Persistent monitoring and construction of a baseline record of temporal variability is essential for distinguishing normal variations from anomalous variations. Without context information the system cannot “think and learn”; it can only apply static criteria to newly collected data in the analysis pipeline. Context information for robotic instrumentation is best provided by a Virtual Observatory (VO) that is constructed from the instrument’s own observations. The VO can then store both the measurements and learned experience (derived information and relations found by ML algorithms) as well as act as a platform for data mining and knowledge extraction. But to maximize adaptability of the anomaly detection software and scientific utility of the VO, it must have a powerful interface that enables extensive data mining and integration with other VO databases.

The positions of objects in the SkyDOT Virtual Observatory brighter than 11^th magnitude (~500,000 objects) plotted in an equal area projection. Each object has a measured time history composed of typically a few hundred measurements spread over at least a one year baseline.

The massive data volumes generated by the new generation of time domain instrumentation and the need to access that information rapidly will require the development of new, cost effective, massive storage systems.

For more information on the database and virtual observatories, see SkyDOT.

The Database Hardware

The Thinking Telescope / SkyDOT database will be contained in a multi-terabyte spinning disk array. This system will be designed and implemented by Mike Warren and Przemek Wozniak. The system will be be similar to a Beowulf cluster but built with off-the-shelf components. It will have high speed performance and fast access rates, but at a minimum of cost to the project.