Doppler Integration

Purpose

The Doppler decisioning engine maintains a dynamic map of domain-to-optimal-CDN provider, on a per ASN basis, changes to which are pushed to DNS providers on a regular basis. However, since DNS propagation can be slow, the updated mapping also is made available via Redis cache by the Doppler Cache Writer service, which transforms a frequenly-updated list of Doppler CDN mappings (stored as a flat file of JSON objects, in cloud storage) into a Redis-cache representation of those mappings. The Redis-cache entries are then used by Bakery to perform direct domain-replacement, thus bypassing problems related to slow DNS propagation.

Flat File Layout

The flat-file contains an array of JSON object, i.e.:

[{
   "<From-Host Key>":{
      "ToHosts":{
         "<To-Host-Key-1":"<To-Host-Name-1>",
         …
         "<To-Host-Key-n>":"<To-Host-Name-n>"
      },
      "Decision":[["<Location-1>", "To-Host-Key-i"],...,["<Location-m>", "<To-Host-Key-j>"]]
   },
   …
},
...]

Simple example:

[{
   "87debgha6d41.airspace-cdn.cbsivideo.com":{
      "ToHosts":{
         "a":"87debgha6d41.airspace-a.cbsivideo.com",
         "f":"87debgha6d41.airspace-f.cbsivideo.com"
      },
      "Decision":[
         [
            "AU:NSW:4804",
            "a"
         ],
         [
            "US:OH:7018",
            "f"
         ]
      ]
   }
}]

Redis Representation

For each JSON object in the flat-file, Doppler Cache Writer creates <key>: <value> pairs in Redis of the form <from-host>___<ASN>: <to-host>. The above JSON object would produce following Redis key/value pairs:

87debgha6d41.airspace-cdn.cbsivideo.com___AU:NSW:4804: 87debgha6d41.airspace-a.cbsivideo.com
87debgha6d41.airspace-cdn.cbsivideo.com___US:OH:7018: 87debgha6d41.airspace-f.cbsivideo.com

Location Header

Bakery requests may contain an optional X-Geo-Asn header, with contains region + ASN information for the requesting client, i.e. the header/value X-Geo-Asn: US:MA:7922 identifes a request from Massachusetts, US by a client in ASN 7922

Origin Subsitution

Prior to requesting a manifest, the manifest-origin (i.e. mediag2481ed.airspacecdn.cbsivideo.com) is combined with the value of the X-Geo-Asn (if present), to form a Redis lookup key, i.e.:

mediag2481ed.airspacecdn.cbsivideo.com___US:MA:7922

If this key exists in Redis, then the value of that key (i.e. mediag2481ed.airspace-f.cbsivideo.com, in the above example) is substiuted for the origin-host, for for request.