I got pulled into optimizing some ancient C code. This was pre-ANSI C. This first thing I found was that there was floating-point function being called repeatedly in some loops with identical arguments, so I added some code near the top of the file that was something like this:

static float floating_point_function_table[20];
static int floating_point_function_table_initialized = 0;

static void init_floating_point_function_table()
{
    int i;
    if (floating_point_function_table_initialized)
        return;
    for (i = 0; i < 20; i++)
        floating_point_function_table[i] = floating_point_function((float) i);
}

When I tried it out, it was a lot faster. After replacing the function_point_function() calls with the table lookups, the loops, which did some other stuff as well, ran in about one sixth the time. But the results were garbage. And I was baffled for at least an hour. The function looked something like this:

float floating_point_function(arg)
  float arg;
{
    ...
}

Since this was ancient code, it was implicitly declared to return int when it was being called in init_floating_point_function_table(), so I moved init_floating_point_function_table() below the definition of floating_point_function(), and it worked.

I also discovered a bunch of loops like this:

    for (i = 0; i < SOME_CONSTANT/SOME_CONSTANT_ARRAY[some_parameter]; i++) ...

in code that stood out when profiling, and declared a new int variable at the top of the function:

    int SOME_CONSTANT_OVER_SOME_CONSTANT_ARRAY_some_parameter;
   SOME_CONSTANT_OVER_SOME_CONSTANT_ARRAY_some_parameter = SOME_CONSTANT/SOME_CONSTANT_ARRAY[some_parameter];

and changed the loops to:

    for (i = 0; i < SOME_CONSTANT_OVER_SOME_CONSTANT_ARRAY_some_parameter; i++) ...

and, surprisingly, even when compiled with full optimization, this change caused this section of code to run in less than half the time.