Warm the milk in a pan, remove the pan from the heat. Whisk the egg yolks and sugar in a bowl. Pour half of the milk into the egg yolks, whisk well. Pour the mixture back into the pan and bring to just below the boil over a medium heat, stirring constantly until the mixture thickens and turns creamy. Pour the mixture through a sieve into a measuring cup, setting aside approx. 150 ml. Pour the remainder of the mixture into the ice lolly moulds, freeze for approx. 30 mins.
Pour the coffee, sugar and water into a measuring cup. Using the whisk on a hand mixer, whisk for approx. 10 mins. until stiff. Pour in the reserved egg & milk mixture, leave to cool. Spread the mixture on top of the frozen ice cream, create a slight marble effect using a teaspoon, insert the ice lolly sticks, freeze for approx. 7 hrs.