Preheat oven to 350 degrees. Line baking sheet with parchment paper or silicone baking mat.
Use mixer to beat butter until lightened, about 2 minutes.
Add in powdered sugar, milk, vanilla and almond extracts. Beat until smooth, scraping down sides as needed.
Add egg yolks and beat until combined.
In a small bowl, stir together flour and salt. Gradually beat into butter mixture, about a 1/2 cup at a time, until well incorporated. Scrape down sides as needed.
Place dough into cookie press and squeeze onto cookie sheet. These cookies can be about an inch apart as they do not spread much. If you have trouble getting the dough to stick, you can place it in the refrigerator for a 5-10 minutes if it's too soft.
Bake until just golden, about 10-12 minutes. Remove from oven and let cookies cool completely before decorating.
To decorate: In a small bowl, mix together water, powdered sugar and vanilla extract. Work in groups of about 4-6 cookies. Brush a small amount over top of cookies and then immediately sprinkle with colored sugar or other sprinkles of your choice.