The computing machine Z3, built by Konrad Zuse between 1938 and 1941, could execute only fixed sequences of floating-point arithmetical operations (addition, subtraction, multiplication, division, and square root) coded in a punched tape. An interesting question to ask, from the viewpoint of the history of computing, is whether or not these operations are sufficient for universal computation. In this paper, I show that, in fact, a single program loop containing these arithmetical instructions can simulate any Turing machine whose tape is of a given finite size. This is done by simulating conditional branching and indirect addressing by purely arithmetical means. Zuse's Z3 is, therefore, at least in principle, as universal as today's computers that have a bounded addressing space. A side effect of this result is that the size of the program stored on punched tape increases enormously.