Fossilization is extremely rare to occur, as it requires specific conditions to be met. It's estimated only a few percent of species that have existed have been discovered this way. Of the ones that were, there are mostly just a few exemplars, with the exception of mollusks and other maritime life with shells.

So of all the biomass that existed, almost none ended up as fossils.

Not to mention that the majority of oil is probably from plant matter, not animal matter.