The numbers are for which total card score the enemy will be in.
5 is for those with a card-score of 0-9,999; if you win you get 1 gacha ticket*
4 is for those with a card-score of 10,000-29,999; if you win you get 2 tickets*
3 is for those with a card-score of 30,000-59,999; if you win you get 3 tickets*
2 is for those with a card-score of 60,000-149,999; if you win you get 4 tickets*
1 is for those with a card-score of 150,000 or above; if you win you get 5 tickets*
Yes, they may have a lower score than you do, but the right kind of card can cancel you out anyway. Green beats Purple, Purple beats Pink, Pink beats Green. The winning type's cardscore is multiplied by 3, so even a silver can beat a gold sometimes.
Any type versus it's own is equal, and equal card-scores go to the attacker. (Watch out for Ex levels though; you may have the best card in it's class, but a second best with maxed Ex levels can still have a higher score)
*5 Tickets are required for one gacha spin. (3 tickets are sent to your inbox for every win you didn't initiate)
Affinity will tell you what cards the enemy has in their deck. (kinda)
You have Green (Moe), Pink (Beauty), and Purple (Lust). If the enemy has 3 or more of a type of card, that type is their affinity. If they don't, they have the black one.
That said, if you have a full deck of Green (Moe) cards, you may want to select Lust enemies to go after since you know at least 3 of their cards are easy wins. The issue here is if they only have 3, and their friends card is a better Moe in that spot, or Pink (Beauty) card, it's a lot tougher to beat them.