Google Tag Manager is a new, free service from Google that helps you set up and manage your web site code snippets. Tags, as Google call them, are code snippets like the Google Analytics tracking code and more. You can add any type of tags to the tag manager, not only Google related ones.
What happens is that Google will provide you with a placeholder tag that is used to enter the other tags. The placeholder should be added in your Joomla template's index file, just after the starting BODY tag.
Most of the time, the template index file is the index.php file in the /templates/yourtemplatename/ folder. Some template frameworks use a different structures, so make sure to check the documentation.
I recommend you take a look at the video to learn more about the Google Tag Manager. I know it will save web site owners a lot of time!