Download all the alert settings for a given application as a YAML formatted file.