I was trying to CAML query a document library, but my returned data set had all the items from the library.

Bad query:

$camlQuery = '<Query><Where><Eq><FieldRef Name="ID" /><Value Type="Text">1624</Value></Eq></Where></Query>'

Good query:

$camlQuery = '<Where><Eq><FieldRef Name="ID" /><Value Type="Text">1624</Value></Eq></Where>'

Removing <query> and </query> from the string fixed the CAML query.