Reading up a bit on the history of swing music at Wikipedia, it apparently developed out of 1920s & '30s jazz. "The name derived from its emphasis on the off-beat, or nominally weaker beat." Not being a musician, I don't really know what that means, but I have been called off-beat before, so I've got that going for me.
Wikipedia explains the "off-beat" like this:
In typical Western music 4/4 time, counted as "1 2 3 4, 1 2 3 4...", the first beat of the bar (downbeat) is usually the strongest accent in the melody and the likeliest place for a chord change, the third is the next strongest: these are "on" beats. The second and fourth are weaker—the "off-beats". Subdivisions (like eighth notes) that fall between the pulse beats are even weaker and these, if used frequently in a rhythm, can also make it "off-beat".
There are sound samples there if you want to hear the difference.
Anyway, the French Romani jazz musician Jean "Django" Reinhardt (1910-1953) picked this up and developed what is called gypsy jazz or gypsy swing. His band was called the Quintette du Hot Club de France, so some call his style hot club.
Here's one of his famous swing pieces.