By removing the ability to see current language and switch it (if there are more than one) you solved only a part of problem. You should also look at the keyboard layouts and fix if need. To do this, go to Language and region, [Your language] (currently French), click ... on the right and select "Language options". There in the "Keyboards" section look at the list of installed keyboards and remove those which you don't use. If there is only one keyboard layout in this list and only one preferred language, the language indicator will not be shown on the desktop without any actions posted above.