Error comparison for first derivative approximations

We will compare the errors in approximating the derivative of sin(x) at x=1, using the forward difference, backward difference, and centered difference approximations.

First, define the function f(x)=sin(x). Also define the three approximations.

f[x_]:=Sin[x];
forward[x0_, Δx_]:=(f[x0+Δx]-f[x0])/Δx;
backward[x0_, Δx_]:=(f[x0]-f[x0-Δx])/Δx;
centered[x0_, Δx_]:=(f[x0+Δx]-f[x0-Δx])/(2Δx);

Next, define the error for each approximation of f'((Subscript[x, 0])). This requires computing the exact value of f'((Subscript[x, 0])).

errforward[x0_,Δx_]:=Abs[forward[x0,Δx]-f'[x0]];
errbackward[x0_,Δx_]:=Abs[backward[x0,Δx]-f'[x0]];
errcentered[x0_,Δx_]:=Abs[centered[x0,Δx]-f'[x0]];

We will compute four approximations of each type, using the following values of Δx:

steps={.1,.01,.001,.0001}

Now compute the errors for each approximation and print them in a table.

TableForm[Table[{errforward[1,Δx],errbackward[1,Δx],errcentered[1,Δx]},{Δx,steps}],TableHeadings->{steps,{"forward","backward","centered"}}]

Note that when we divide Δx by 10, the errors for the forward and backward approximations are also divided by about 10, but the error for the centered approximation is divided by about 100.

Extend the table for a few more rows. What do you observe?

steps={0.1,0.01,0.001,0.0001,0.00001,0.000001}

TableForm[Table[{errforward[1,Δx],errbackward[1,Δx],errcentered[1,Δx]},{Δx,steps}],TableHeadings->{steps,{"forward","backward","centered"}}]

Something is going wrong for the the centered approximation!

The problem is round-off error. Round-off error is a big concern when doing computations with decimal numbers.

Precision[errcentered[1,0.000001]]

MachinePrecision means that Mathematica is using floating-point arithmetic, as implemented in the computer processor. For most computers, this means 64-bit “double precision” floating-point numbers, which use 53 bits of memory to store significant digits. This amounts to almost 16 decimal digits of precision, as the following commands show:

$MachinePrecision

N[53Log[10,2]]

If we avoid typing decimals, Mathematica will attempt to compute using arbitrary precision arithmetic. However, arbitrary precision calculations may take much more time than machine precision calculations. For example:

errcentered[1,10^(-6)]

The formula above is exact, but probably not helpful. We can ask Mathematica to convert it to a decimal number with, say, 20 digits:

N[errcentered[1,10^(-6)],20]

That' s better! Lastly, let’s see what Mathematica says is the precision of this calculation:

Precision[errcentered[1,10^(-6)]]