You go to church to pray and worship God, not to see what someone is wearing.
We are not to judge others. Maybe that is all the person has to wear. Not everyone has dress clothes, not everyone can afford them.
There are more important things in this world to worry about.
As far as the priest saying something about dress, he is probably just happy people are coming to church.
Wearing jeans or dress clothes will not change church, only you can by what you put in it and what you take away from it.