Many other indicators are commonly present, including ones that can even tell you the angle of the depositional surface at the time ("geopetal structures"), "assuming" that gravity was "down" at the time, which isn't much of an assumption :-).
In more complicated situations, like in a mountain belt, there are often faults, folds, and other structural complications that have deformed and "chopped up" the original stratigraphy.
The layers of rock are known as "strata", and the study of their succession is known as "stratigraphy".
However, note that because of the "principle of cross-cutting relationships", careful examination of the contact between the cave infill and the surrounding rock will reveal the true relative age relationships, as will the "principle of inclusion" if fragments of the surrounding rock are found within the infill.
Cave deposits also often have distinctive structures of their own (e.g., spelothems like stalactites and stalagmites), so it is not likely that someone could mistake them for a successional sequence of rock units. Each of them is a testable hypothesis about the relationships between rock units and their characteristics.
As an example of how they are used, radiometric dates from geologically simple, fossiliferous Cretaceous rocks in western North America are compared to the geological time scale.
To get to that point, there is also a historical discussion and description of non-radiometric dating methods.
his document discusses the way radiometric dating and stratigraphic principles are used to establish the conventional geological time scale.