Here is a link to MSDN detailed information on Protected Mode, how it works, how to properly configure it and work with it:
http://msdn2.microsoft.com/en-us/library/bb250462.aspx
I am sure you will be able to locate the instructions on turning Protected Mode on/off in these links. You may also learn a lot of important new information regarding this feature.
Good luck and much success, have a nice day.
*edit* after rereading some of the information at the link I provided, I realized the data might not apply in your situation, that you may require simpler information.
In order to turn Protected Mode on/off you need to do so for each particular internet zone. Go into Internet Options and click on the Security Tab. Then, click on one of the Internet Zones, such as the Internet, then click Settings and click the check box in front of Protected Mode. Do this for each Zone you wish to turn Protected Mode ON for, and then click Apply and then OK. Close IE and then reopen IE, and Protected Mode should now be set at the levels you choose.
*edit* After thinking on your issue for a while, I decided to add a few comments and suggestions.
If you continue to have issues with Protected Mode, this could signal a deeper issue with either IE or your OS. Protected Mode is based on the underlying new upgraded security features in Vista. This is why IE7 for XP does not have Protected Mode options.
So, if you continue to find it impossible to switch back on, then you may need to look at different areas, and it is possible your IE instance has become corrupted. When a program becomes corrupt all sorts of odd behavior can ensue. This is due to all the different files which make up the different programmed components in any application and/or OS.
For example, when you begin experiencing issues with Office, the first step is to access the Detect and Repair option within the Office applications Help Menu. If your Office files are located on your hard disk the program quickly checks all the files which make up that Office application, and if it locates a corrupt file it replaces it with a copy stored on the HD. If it finds a corrupt file or files and the Office setup files are not stored on the HDD, then the Detect and Repair process asks for the disk to be inserted so it can scan and locate the disk to exchange with the corrupted ones.
The same goes for any other software application on a system. The first step in fixing an issue is always to remove it and then reinstall it. The act of removal and reinstallation recopies the neccassary files and Voila! the issue is fixed. If your Windows Live Messenger begins to not display your contacts, or menus come up missing, etc, you download a new instance of Messenger from its download webpage, remove the currently installed instance, and then do a quick clean up of files with a utility such as CCleaner, and then install the new instance from the download package on the desktop. Fixed.
So, I suggest you remove this instance of IE and then download and install another instance which you can obtain from the Microsoft Downloads website. Here is a link for the Downloads area:
http://www.microsoft.com/windows/products/winfamily/ie/default.mspx
If after removing your current instance of IE you still experience issues, I strongly suggest you visit a Microsoft Newsgroup for IE7 and ask for assistance from the experts there. Now, the individuals who offer out assistance at such newsgroups do it on a volunteer basis, so it may take a bit of time to get back to you and then get your issue solved. Here are a few tips on how to use the Newsgroups:
First, don't post the same question to multiple newsgroups. This will not get you help any faster and it angers the regulars of the groups. These groups are manned by a certain regular MVP's who will know about cross posting. If you honestly believe your question should be posted at more than one group, do it via the option in the post formatting window. This option will post your question to the groups you specify at one time, and thus will be viewed more favorably. However, the question must be relevent in each group it is posted to.
For example, if you are asking about IE7, you might post to the IE7 Newsgroup, and also the Security Newsgroup, because it is an issue on security as well as an issue on how to use and configure IE. However, you wouldn't post your IE question to the Networking Newsgroup as it wouldn't be relevent in that group.
Next, make sure you list your system configuration at the beginning of your post. It is difficult to answer a question if you don't know what OS version is in use, what type of processor, how much RAM, how much space is left in your HDD, etc. The manufacturer and version of your system would be needed too. After this write down what your issue is, and the steps you have taken thus far to correct it for yourself. The members who respond like to help those who help themselves first and show an inititive to at least attempting to research the issue.
For example, you would say you have an issue with being unable to turn Protected Mode back on, that you have visited the Vista Help and How-to for information on the issue, and were unable to locate anything. Then go on to state you have searched Online via Google and Live Search without luck. (All this is true as I did it for you, in this case! lol) You can wind up by explaining how you asked at Yahoo! Answers to see if anyone else has experienced this issue, and somebody there steared you to the Newsgroups.
Please be patient, and even if somebody else is rude to you don't allow yourself to respond in kind. That is the quickest method of not receiving assistance. The members who work in the groups the most, the experts, correct each other's behavior. So, if somebody is rude, just suck it up, and don't be surprised if you later read a post which tells the person they shouldn't talk that way to those asking for help. This doesn't always happen, it depends on the degree of rudeness, but the MVP's do not want members to go away, or the group falls apart. Here is a link to the Newsgroup area:
http://www.microsoft.com/technet/community/newsgroups/dgbrowser/en-us/default.mspx?query=Internet+Explorer+7&cboDiscussionGroup=en-us-technet-appcenter%3A%3Aen%3AUS%3A%3A328BAFD2-1A81-4558-B1DE-B6EB49F31B7E%3A%3A&=Go&dg=&cat=en-us-technet-appcenter&lang=en&cr=US&pt=&catlist=328BAFD2-1A81-4558-B1DE-B6EB49F31B7E&dglist=&ptlist=&exp=&sloc=en-us
One last item: You will need a .Net Passport to participate in Microsoft TechNet Newsgroups. If you have a Hotmail, or Windows Live email account, the email address and password to access it is your passport. If you do not have a passport, you can easily and quickly sign up for one at the logon interface webpage. When you click to post a question the logon page will be displayed. Make sure you put the Newsgroup into your favorites or Links, so you can find it again if you accidently lose it. Now, on the logon page you will see a link to sign up for a Passport if you don't have one. Don't be fooled, you can attach any email address to your passport, it doesn't have to be Hotmail or Windows Live Mail. Any existing email account will work. Just walk through the Passport Wizard and use your current email and password.
You can also connect your User Account to your Passport. When you do this you can log onto all of your Microsoft resources which require membership or a password at one time, rather than logging in several times in a row. This is very handy, and I encourage your doing this if you feel comfortable doing so. You do this by opening up the User Account applet in Windows and clicking on the link to create a Passport. I am usnsure of the exact wording without actually opening it, but it is there all the same. You can do this from your desktop, before going to the Newsgroups and posting if you prefer.
Also, please search the Newsgroup for similar questions before posting yours. You may be surprised at the wealth of information at your fingertips due to the other posts. I often go to the Newsgroups simply to read up on issues, and/or learn new things about my system. If I want to learn about networking I go to the networking group, for administration I go to that group and do on and so forth. I often don't even post a question, simply read the ones already there.
I know I have given you a great deal of information here. I wish you the best in finding a solution to your issue. Please have a nice day.