Excellent question! (any question that makes me think)
I have been doing it wrong.This is how I did it just now.
I went to the doc and settings folder, application data Mozilla folder and found firefox and deleted it. Went to the c:\program files folder and found firefox under mozilla.org and deleted the firefox folder. Went to the D:\program files folder and deleted firefox folder. Ran Registry first aid and cleaned all refs it found. Now have no firefox stuff on the computer.
Then I unzipped the install file to D:\Program Files. Found firefox.exe and sent it to the desktop as a shortcut. Opened that and firefox asked me if I wanted to import from Internet Explorer, Netscape or Mozilla. I selected none and firefox opened with a new profile and waited for me to set my prefs etc.
As I remember it, when I did this before your excellent question, I was not asked the import question. It used the default profile located on C: drive that was there from the old version.
I now have D:\Program Files\Firefox and C:\Documents and settings\william\application data\mozilla\firefox\profiles with a new profile.
This is what I suppose would be called a clean installation of Firefox. Right Ramona?