Yes, you have malware on your computer if you are getting those type of pop-ups. Try downloading one or more of these free products and run full system scans.
AntiSpyware Applications
SuperAntispyware
http://superantispyware.com/
Spybot S&D (free version requires manual updating)
http://www.safer-networking.org/en/download/
Adaware from Lavasoft (free version requires automatic updating)
http://www.lavasoftusa.com/products/ad_aware_free.php
(These programs can be run in Safe Mode after installation and updating)
If you are successful in getting the malware removed, you may want to also download a spyware blocker. This is free and requires you to check for and enable updates manually once a week or so. It uses no system resources. It blocks spyware before it installs.
SpywareBlaster (inoculates, not a spyware removal tool)
http://www.javacoolsoftware.com/spywareblaster.html