All the goals came before half time at Rugby Park, with the Germans appearing happy to hang on for a point after the break.
Benjamin Lauth gave the visitors the lead after ten minutes bundling the ball home from close range after Thomas Hitzlsperger's header had been saved.
But the Scots hit back five minutes later when Kerr released Peter Canero, who fed Simon Lynch to score from close range.
Ten minutes later Caldwell bundled the ball home from a Stephen Crainey corner to put the home team ahead.
But the Germans completed the scoring shortly before the break with Hanno Balitsch earning them a point with another close-range effort.
The result leaves the two sides level on points at the top of their European Championship qualifying group, with the Germans having a game in hand.