This advice is entirely based upon my short visit. Therefore, I will only discuss rural parts of Southern France. Paris and things like that shall not be discussed since I have never been there.

Visit in a non-tourist season
See the Alps
Go to small towns instead of the bigger, famous cities
Indulge on local food and avoid restaurants marketed towards tourists
Learn some French (it will make things much easier)
Try cheese and wine (in this case, they do live up to the hype)
Go to bakeries. You will never be disappointed.
See medieval buildings and architecture
Public transportation can be expensive and confusing. Prepare yourself.
Go to a local open market and buy a crossiont. I know it’s stereotypical but trust me on this.