As with many forms of wrestling around the world, the roots of Sumo are lost in prehistory. Sumo is mentioned in some of the earliest texts in Japan, under its earlier name Sumai, from the 8th century A.D. However, these early forms would not be Sumo as it is known today, as in many cases the wrestling had relatively few rules and unarmed fights to the death were still referred to as 'Sumo'.

In addition to its use as a trial of strength in combat, it has also been associated with Shinto ritual, and even today certain shrines carry out forms of ritual dance where a human ceremonially wrestles with a kami (a Shinto 'spirit' or 'god'). It was an important ritual at the imperial court. Representatives of each province were ordered to attend the contest at the court and fought. They needed to pay for their travels by themselves. The contest was called Sumai no sechie (Party of Sumai).

Over the rest of Japanese recorded history Sumo's popularity has changed according to the whims of its rulers and the need for its use as a training tool in periods of civil strife. The form of wrestling combat probably changed gradually into one where the main aim in victory was to throw your opponent. The concept of pushing him out of a limited defined area came some time later.

It is believed that a ring, defined by more than the area given to the wrestlers by spectators, came into being in the 16th century as a result of a tournament organized by the then principal warlord in Japan, Oda Nobunaga, but at this point wrestlers would wear loose loincloths, rather than the much stiffer mawashi of today. During the Edo period, wrestlers would wear a fringed kesho-mawashi during the bout, as opposed to the merely ceremonial role they hold today. Much of the rest of the development came in the early Edo period to give the sport its current form.

It is worth noting that nations adjacent to Japan, having shared some cultural traditions, also feature styles of traditional wrestling that bear some resemblance to Sumo. Notable examples include Mongolia, the birthplace of Asashoryu (the current Yokozuna), and Korea, where a similar sport called Ssireum is popular.