Skip to main content
SearchLogin or Signup

Macduff: An astronomical example

Published onJul 14, 2020
Macduff: An astronomical example

An observatory network might have a few million observations a night. Most telescopes capture much more data than they preserve; they make their best effort to save what might be useful to their own future work or to others who could use something they’ve seen in the context of othe robservations.

As a result, astronomical ‘event brokers’ listen to the observation streams of a network and filter for potential events that different audiences [usually: other observatories] might be interested in, in order to co-associate further observations or annotations.

What does the network need to do to turn a subset of observations into UL packages?

What does a broker need to do to maintain its own registry (possibly large, largely unused, speculative datasets — costs of storage covered by its brokerage service) for original + filtered packages?

Example + background

Telescopes are real-world data sources with

  • hardware type

  • configuration

  • calibration functions tied to the {type, config}

  • calibration data updated at least nightly, tuning those functions

    • weather data, if terrestrial, informing this

  • location

An observation from a scope has

  • target location

  • angular magnification

  • field angle (field of view)

  • time

  • duration (time over which light is integrated)
    this allows things to be somewhat fuzzy: you could compile many observations of different durations from a raw video- or image-stream.

  • implicit noise: mechanical, electrical, astronomical, cosmic rays.

An event is the appearance of something new in the sky. It has

  • time of first observation (a lower bound)

  • duration (if transient)

  • classification

  • weight / likelihood of being a real event, observed from most locations that happened to observe the same target (concept not standardized)

An event stream from a set of sources is a feed of timestamped events that cross some threshhold of likelihood to be interesting.

Past examples: VOEventNet (07), SkyAlert (09), CRTS (09), automated classification of transients (11).

Current examples:

  • the Large Synoptic Survey Telescope (’15) : a raw feed of 1M potential transient events a night; a simple ‘event broker’ to help flag events of interest to common users; and access to the full stream (for a fee) granted to external brokers who flag different subsets for different audiences.

  • the Antares (+)and GROWTH projects — both developing machine-learning tools for automating filtering + classifying events, allowing for automated telescopes to choose where to look based on the results

  • Maximizing Science in the LSST Era” (pp.138-40, 145-9) has many detailed examples, including data sharing tools, community brokers w/ local data filters, cross-matching of alerts, and the need for common protocols of communication b/t brokers and other data services (e.g., SIMBAD and NED)

In addition to communication protocols for managing a shared distributed feed, aligning events across different scopes requires cross-calibration.

  • Where the scopes use the same config, just a comparison of parameters.

  • Where setups are different, a mapping function that projects both into comparable space+time coordinates.


Inputs. Say we have 6 telescopes in our network, Ta - Tg,
run by Anubis, Balor, Charon, Dante, Eurydice, and Frank.

Ta and Tb are the same hardware in different locations.
Tc is different hardware with roughly the same settings.
Td is a scope capturing a continuous imagestream, converted later into chunked observations.
Te is an orbital scope.
Tf is a virtual-aperture scope (combined data from multiple scopes on different sides of the planet)


There are at least four main outputs desired from this network: existence, alignment, combination, and replication.

Existence - What events have been observed in this region / within this time-range?

Alignment - Did anyone else observe something close to this {time/place/classification}? Did anyone observe something sub-threshhold?

Combination - An aggregate feed from many sources. Combined events: events where the combined data from multiple sources/spectra pass a combined threshhold. A combined feed of only combined-event data.

Real-time replication - A prioritized feed of recent events that merit further observation, often time-sensitive [transient events]. A streamlined feed of high-priority events and their followups, perhaps segmented or filtered by the types of scopes that could respond [which spectra are most interesting to observe].


No comments here