Desert adventure, want to go rock hunting?

i live in the san fernando valley, and remember when i was a lot younger my parents took my brother and i to what could have been a resort under development or housing, ANYWAY, the thing is, its somewhere in desert like area, and i remember specifically that we went walking into the desert and where collecting loads and loads of crystals, pink crystals, and other semi precious stones. i'd love to do that again.....ANYONE KNOW WHERE THIS PLACE IS? or what areas around southern california that are dry and arid where i can walk freely into and find such rocks? asking dad is not possible :-/ my brother and i are all thats left, and we would like to relive some old memories. thank you so much for your suggestions. any place is fine. or if you have any other suggestions where one might have some natural adventure like this, would be great.