major rework of backend features

* use flask for easier url handling and tidyness
 * create radio-browser.info and vtuner api classes
 * add support for more vtuner logic (logos, info messages, search, buttons, etc.)
 * use radio-browser.info index and search
 * prepare for python packaging

README.md

@@ -3,7 +3,7 @@
 # YCast
 
 YCast is a self hosted replacement for the vTuner internet radio service which many AVRs use.
-It emulates a vTuner backend to provide your AVR with the necessary information to play self defined categorized internet radio stations.
+It emulates a vTuner backend to provide your AVR with the necessary information to play self defined categorized internet radio stations and listen to Radio stations listed in the [Community Radio Browser index](http://www.radio-browser.info).
 
 YCast is for you if:
  * You do not want to use a proprietary streaming service
@@ -41,6 +41,7 @@ Go ahead and test it with yours, and kindly report the result back :)
 Python version: `3`
 
 Python packages:
+ * `flask`
  * `PyYAML`
  
 ## Usage
@@ -48,14 +49,13 @@ Python packages:
 YCast really does not need much computing power nor bandwidth. It just serves the information to the AVR. The streaming
 itself gets handled by the AVR directly, i.e. you can run it on a low-spec RISC machine like a Raspberry Pi.
 
-1) Create your initial `stations.yml` and put it in the same directory as `ycast.py`. The config follows a basic YAML structure (see below).
+You need to create a manual entry in your DNS server (read 'Router' for most home users). `vtuner.com` should point to the machine YCast is running on. Alternatively, in case you only want to forward specific vendors, the following entries may be configured:
-2) Create a manual entry in your DNS server (read 'Router' for most home users). `vtuner.com` should point to the machine YCast is running on. Alternatively, in case you only want to forward specific vendors, the following entries may be configured:
 
   * Yamaha AVRs: `radioyamaha.vtuner.com` (and optionally `radioyamaha2.vtuner.com`)
   * Onkyo AVRs: `onkyo.vtuner.com` (and optionally `onkyo2.vtuner.com`)
   * Denon/Marantz AVRs: `denon.vtuner.com` (and optionally `denon2.vtuner.com`)
 
-3) Run `ycast.py`.
+If you want to use the 'My Stations' feature besides the global radio index, create a `stations.yml` and run YCast with the `-c` switch to specify the path to it. The config follows a basic YAML structure (see below).
 
 ### stations.yml
 ```
@@ -68,6 +68,14 @@ Category two name:
   Fourth awesome station name: fourth.awesome/station/URL
 ```   
 
+### Running
+
+You can run YCast by using the built-in development server of Flask (not recommended for production use, but should(tm) be enough for your private home use): Just run the package: `python -m ycast`
+
+Alternatively you can also setup a proper WSGI server.
+
+ -- TODO: WSGI stuff
+
 You can also have a look at the provided [example](examples/stations.yml.example) to better understand the configuration.
 
 

ycast.py

@@ -1,105 +0,0 @@
-#!/usr/bin/env python3
-
-import os
-import sys
-import argparse
-from http.server import BaseHTTPRequestHandler, HTTPServer
-import xml.etree.cElementTree as etree
-
-import yaml
-
-YCAST_LOCATION = 'ycast'
-
-stations = {}
-
-
-def get_stations():
-    global stations
-    ycast_dir = os.path.dirname(os.path.realpath(__file__))
-    try:
-        with open(ycast_dir + '/stations.yml', 'r') as f:
-            stations = yaml.load(f)
-    except FileNotFoundError:
-        print("ERROR: Station configuration not found. Please supply a proper stations.yml.")
-        sys.exit(1)
-
-
-def text_to_url(text):
-    return text.replace(' ', '%20')
-
-
-def url_to_text(url):
-    return url.replace('%20', ' ')
-
-
-class YCastServer(BaseHTTPRequestHandler):
-    def do_GET(self):
-        get_stations()
-        self.address = 'http://' + self.headers['Host']
-        if 'loginXML.asp?token=0' in self.path:
-            self.send_xml('<EncryptedToken>0000000000000000</EncryptedToken>')
-        elif self.path == '/' \
-                or self.path == '/' + YCAST_LOCATION \
-                or self.path == '/' + YCAST_LOCATION + '/'\
-                or self.path.startswith('/setupapp'):
-            xml = self.create_root()
-            for category in sorted(stations, key=str.lower):
-                self.add_dir(xml, category,
-                             self.address + '/' + YCAST_LOCATION + '/' + text_to_url(category))
-            self.send_xml(etree.tostring(xml).decode('utf-8'))
-        elif self.path.startswith('/' + YCAST_LOCATION + '/'):
-            category = url_to_text(self.path[len(YCAST_LOCATION) + 2:].partition('?')[0])
-            if category not in stations:
-                self.send_error(404)
-                return
-            xml = self.create_root()
-            for station in sorted(stations[category], key=str.lower):
-                self.add_station(xml, station, stations[category][station])
-            self.send_xml(etree.tostring(xml).decode('utf-8'))
-        else:
-            self.send_error(404)
-
-    def send_xml(self, content):
-        xml_data = '<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>'
-        xml_data += content
-        self.send_response(200)
-        self.send_header('Content-type', 'text/html')
-        self.send_header('Content-length', len(xml_data))
-        self.end_headers()
-        self.wfile.write(bytes(xml_data, 'utf-8'))
-
-    def create_root(self):
-        return etree.Element('ListOfItems')
-
-    def add_dir(self, root, name, dest):
-        item = etree.SubElement(root, 'Item')
-        etree.SubElement(item, 'ItemType').text = 'Dir'
-        etree.SubElement(item, 'Title').text = name
-        etree.SubElement(item, 'UrlDir').text = dest
-        return item
-
-    def add_station(self, root, name, url):
-        item = etree.SubElement(root, 'Item')
-        etree.SubElement(item, 'ItemType').text = 'Station'
-        etree.SubElement(item, 'StationName').text = name
-        etree.SubElement(item, 'StationUrl').text = url
-        return item
-
-
-parser = argparse.ArgumentParser(description='vTuner API emulation')
-parser.add_argument('-l', action='store', dest='address', help='Listen address', default='0.0.0.0')
-parser.add_argument('-p', action='store', dest='port', type=int, help='Listen port', default=80)
-arguments = parser.parse_args()
-get_stations()
-try:
-    server = HTTPServer((arguments.address, arguments.port), YCastServer)
-except PermissionError:
-    print("ERROR: No permission to create socket. Are you trying to use ports below 1024 without elevated rights?")
-    sys.exit(1)
-print('YCast server listening on %s:%s' % (arguments.address, arguments.port))
-try:
-    server.serve_forever()
-except KeyboardInterrupt:
-    pass
-print('YCast server shutting down')
-server.server_close()

ycast/__init__.py

@@ -0,0 +1 @@
+__version__ = '1.0.0'

ycast/__main__.py

@@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+import argparse
+import logging
+
+from ycast import server
+
+logging.basicConfig(format='%(asctime)s %(levelname)s: %(message)s', datefmt='%Y-%m-%d %H:%M:%S', level=logging.INFO)
+
+
+def launch_server():
+    parser = argparse.ArgumentParser(description='vTuner API emulation')
+    parser.add_argument('-c', action='store', dest='config', help='Station configuration', default=None)
+    parser.add_argument('-l', action='store', dest='address', help='Listen address', default='0.0.0.0')
+    parser.add_argument('-p', action='store', dest='port', type=int, help='Listen port', default=80)
+    arguments = parser.parse_args()
+    logging.info("YCast server starting on %s:%s" % (arguments.address, arguments.port))
+    server.run(arguments.config, arguments.address, arguments.port)
+
+
+if __name__ == "__main__":
+    launch_server()

ycast/radiobrowser.py

@@ -0,0 +1,92 @@
+import requests
+
+MINIMUM_COUNT_GENRE = 5
+MINIMUM_COUNT_COUNTRY = 5
+MINIMUM_BITRATE = 64
+STATION_LIMIT_DEFAULT = 99
+ID_PREFIX = "RB_"
+
+
+def get_json_attr(json, attr):
+    try:
+        return json[attr]
+    except:
+        return None
+
+
+class Station:
+    def __init__(self, station_json):
+        self.id = ID_PREFIX + get_json_attr(station_json, 'id')
+        self.name = get_json_attr(station_json, 'name')
+        self.url = get_json_attr(station_json, 'url')
+        self.icon = get_json_attr(station_json, 'favicon')
+        self.tags = get_json_attr(station_json, 'tags').split(',')
+        self.country = get_json_attr(station_json, 'country')
+        self.language = get_json_attr(station_json, 'language')
+        self.votes = get_json_attr(station_json, 'votes')
+        self.codec = get_json_attr(station_json, 'codec')
+        self.bitrate = get_json_attr(station_json, 'bitrate')
+
+
+def request(url):
+    headers = {'content-type': 'application/json', 'User-Agent': 'YCast'}
+    response = requests.get('http://www.radio-browser.info/webservice/json/' + url, headers=headers)
+    if response.status_code != 200:
+        print("error")
+        return None
+    return response.json()
+
+
+def get_station_by_id(uid):
+    station_json = request('stations/byid/' + str(uid))
+    return Station(station_json[0])
+
+
+def search(name):
+    stations = []
+    stations_json = request('stations/search?order=votes&reverse=true&bitrateMin=' + MINIMUM_BITRATE + '&name=' + str(name))
+    for station_json in stations_json:
+        stations.append(Station(station_json))
+    return stations
+
+
+def get_countries():
+    countries = []
+    countries_raw = request('countries')
+    for country_raw in countries_raw:
+        if get_json_attr(country_raw, 'name') and get_json_attr(country_raw, 'stationcount') and int(get_json_attr(country_raw, 'stationcount')) > MINIMUM_COUNT_COUNTRY:
+            countries.append(get_json_attr(country_raw, 'name'))
+    return countries
+
+
+def get_genres():
+    genres = []
+    genres_raw = request('tags?hidebroken=true')
+    for genre_raw in genres_raw:
+        if get_json_attr(genre_raw, 'name') and get_json_attr(genre_raw, 'stationcount') and int(get_json_attr(genre_raw, 'stationcount')) > MINIMUM_COUNT_GENRE:
+            genres.append(get_json_attr(genre_raw, 'name'))
+    return genres
+
+
+def get_stations_by_country(country, limit=STATION_LIMIT_DEFAULT):
+    stations = []
+    stations_json = request('stations/search?order=votes&reverse=true&bitrateMin=' + MINIMUM_BITRATE + '&limit=' + str(limit) + '&countryExact=true&country=' + str(country))
+    for station_json in stations_json:
+        stations.append(Station(station_json))
+    return stations
+
+
+def get_stations_by_genre(genre, limit=STATION_LIMIT_DEFAULT):
+    stations = []
+    stations_json = request('stations/search?order=votes&reverse=true&bitrateMin=' + MINIMUM_BITRATE + '&limit=' + str(limit) + '&tagExact=true&tag=' + str(genre))
+    for station_json in stations_json:
+        stations.append(Station(station_json))
+    return stations
+
+
+def get_stations_by_votes(limit=STATION_LIMIT_DEFAULT):
+    stations = []
+    stations_json = request('stations?order=votes&reverse=true&limit=' + str(limit))
+    for station_json in stations_json:
+        stations.append(Station(station_json))
+    return stations

ycast/server.py

@@ -0,0 +1,174 @@
+import logging
+
+import yaml
+from flask import Flask, request, url_for
+
+import ycast.vtuner as vtuner
+import ycast.radiobrowser as radiobrowser
+
+
+PATH_ROOT = 'ycast'
+PATH_CUSTOM_STATIONS = 'my_stations'
+PATH_RADIOBROWSER = 'radiobrowser'
+PATH_RADIOBROWSER_COUNTRY = 'country'
+PATH_RADIOBROWSER_GENRE = 'genre'
+PATH_RADIOBROWSER_POPULAR = 'popular'
+PATH_RADIOBROWSER_SEARCH = 'search'
+
+my_stations = {}
+app = Flask(__name__)
+
+
+def run(config, address='0.0.0.0', port=8010):
+    try:
+        get_stations(config)
+        app.run(host=address, port=port)
+    except PermissionError:
+        logging.error("No permission to create socket. Are you trying to use ports below 1024 without elevated rights?")
+
+
+def get_stations(config):
+    global my_stations
+    if not config:
+        logging.warning("If you want to use the 'My Stations' feature, please supply a valid station configuration")
+        return
+    try:
+        with open(config, 'r') as f:
+            my_stations = yaml.safe_load(f)
+    except FileNotFoundError:
+        logging.error("Station configuration '%s' not found", config)
+        return
+    except yaml.YAMLError as e:
+        logging.error("Config error: %s", e)
+        return
+
+
+# TODO: vtuner doesn't do https (e.g. for logos). make an icon cache
+
+
+@app.route('/', defaults={'path': ''})
+@app.route('/setupapp/<path:path>')
+@app.route('/' + PATH_ROOT + '/', defaults={'path': ''})
+def landing(path):
+    if request.args.get('token') == '0':
+        return vtuner.get_init_token()
+    page = vtuner.Page()
+    page.add(vtuner.Directory('Radiobrowser', url_for('radiobrowser_landing', _external=True)))
+    page.add(vtuner.Directory('My Stations', url_for('custom_stations_landing', _external=True)))
+    return page.to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_CUSTOM_STATIONS + '/')
+def custom_stations_landing():
+    page = vtuner.Page()
+    page.add(vtuner.Previous(url_for("landing", _external=True)))
+    if not my_stations:
+        page.add(vtuner.Display("No stations found"))
+    else:
+        for category in sorted(my_stations, key=str.lower):
+            directory = vtuner.Directory(category, url_for('custom_stations_category', _external=True, category=category))
+            page.add(directory)
+    return page.to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_CUSTOM_STATIONS + '/<category>')
+def custom_stations_category(category):
+    page = vtuner.Page()
+    page.add(vtuner.Previous(url_for('custom_stations_landing', _external=True)))
+    if category not in my_stations:
+        page.add(vtuner.Display("Category '" + category + "' not found"))
+    else:
+        for station in sorted(my_stations[category], key=str.lower):
+            station = vtuner.Station(None, station, None, my_stations[category][station], None, None, None, None, None, None)
+            page.add(station)
+    return page.to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/')
+def radiobrowser_landing():
+    page = vtuner.Page()
+    page.add(vtuner.Previous(url_for('landing', _external=True)))
+    page.add(vtuner.Directory('Genres', url_for('radiobrowser_genres', _external=True)))
+    page.add(vtuner.Directory('Countries', url_for('radiobrowser_countries', _external=True)))
+    page.add(vtuner.Directory('Most Popular', url_for('radiobrowser_popular', _external=True)))
+    page.add(vtuner.Search('Search', url_for('radiobrowser_search', _external=True, path='')))
+    return page.to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_COUNTRY + '/')
+def radiobrowser_countries():
+    countries = radiobrowser.get_countries()
+    page = vtuner.Page()
+    page.add(vtuner.Previous(url_for('radiobrowser_landing', _external=True)))
+    for country in countries:
+        page.add(vtuner.Directory(country, url_for('radiobrowser_country_stations', _external=True, country=country)))
+    return page.to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_COUNTRY + '/<country>')
+def radiobrowser_country_stations(country):
+    stations = radiobrowser.get_stations_by_country(country)
+    page = vtuner.Page()
+    page.add(vtuner.Previous(url_for('radiobrowser_countries', _external=True)))
+    if len(stations) == 0:
+        page.add(vtuner.Display("No stations found for country '" + country + "'"))
+    else:
+        for station in stations:
+            page.add(vtuner.Station(station.id, station.name, ', '.join(station.tags), station.url, station.icon, station.tags[0], station.country, station.codec, station.bitrate, None))
+    return page.to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_GENRE + '/')
+def radiobrowser_genres():
+    genres = radiobrowser.get_genres()
+    page = vtuner.Page()
+    page.add(vtuner.Previous(url_for('radiobrowser_landing', _external=True)))
+    for genre in genres:
+        page.add(vtuner.Directory(genre, url_for('radiobrowser_genre_stations', _external=True, genre=genre)))
+    return page.to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_GENRE + '/<genre>')
+def radiobrowser_genre_stations(genre):
+    stations = radiobrowser.get_stations_by_genre(genre)
+    page = vtuner.Page()
+    page.add(vtuner.Previous(url_for('radiobrowser_genres', _external=True)))
+    if len(stations) == 0:
+        page.add(vtuner.Display("No stations found for genre '" + genre + "'"))
+    else:
+        for station in stations:
+            page.add(vtuner.Station(station.id, station.name, ', '.join(station.tags), station.url, station.icon, station.tags[0], station.country, station.codec, station.bitrate, None))
+    return page.to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_POPULAR + '/')
+def radiobrowser_popular():
+    stations = radiobrowser.get_stations_by_votes()
+    page = vtuner.Page()
+    page.add(vtuner.Previous(url_for('radiobrowser_landing', _external=True)))
+    for station in stations:
+        page.add(vtuner.Station(station.id, station.name, ', '.join(station.tags), station.url, station.icon, station.tags[0], station.country, station.codec, station.bitrate, None))
+    return page.to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_SEARCH, defaults={'path': ''})
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_SEARCH + '<path:path>')
+def radiobrowser_search(path):
+    page = vtuner.Page()
+    page.add(vtuner.Previous(url_for('radiobrowser_landing', _external=True)))
+    # vtuner does totally weird stuff here: TWO request arguments are passed to the search URI
+    # thus, we need to parse the search query as path
+    query = None
+    if 'search' in path:
+        path_search = path[path.find('search'):]
+        query = path_search.partition('=')[2]
+    if not query or len(query) < 3:
+        page.add(vtuner.Display("Search query too short."))
+    else:
+        stations = radiobrowser.search(query)
+        if len(stations) == 0:
+            page.add(vtuner.Display("No results for '" + query + "'"))
+        else:
+            for station in stations:
+                page.add(vtuner.Station(station.id, station.name, ', '.join(station.tags), station.url, station.icon, station.tags[0], station.country, station.codec, station.bitrate, None))
+    return page.to_string()

ycast/vtuner.py

@@ -0,0 +1,108 @@
+import xml.etree.cElementTree as etree
+
+XML_HEADER = '<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>'
+
+
+def get_init_token():
+    return XML_HEADER + '<EncryptedToken>0000000000000000</EncryptedToken>'
+
+
+class Page:
+    def __init__(self):
+        self.items = []
+
+    def add(self, item):
+        self.items.append(item)
+
+    def to_xml(self):
+        xml = etree.Element('ListOfItems')
+        etree.SubElement(xml, 'ItemCount').text = str(len(self.items))
+        for item in self.items:
+            item.append_to_xml(xml)
+        return xml
+
+    def to_string(self):
+        return XML_HEADER + etree.tostring(self.to_xml()).decode('utf-8')
+
+
+class Previous:
+    def __init__(self, url):
+        self.url = url
+
+    def append_to_xml(self, xml):
+        item = etree.SubElement(xml, 'Item')
+        etree.SubElement(item, 'ItemType').text = 'Previous'
+        etree.SubElement(item, 'UrlPrevious').text = self.url
+        etree.SubElement(item, 'UrlPreviousBackUp').text = self.url
+        return item
+
+
+class Display:
+    def __init__(self, text):
+        self.text = text
+
+    def append_to_xml(self, xml):
+        item = etree.SubElement(xml, 'Item')
+        etree.SubElement(item, 'ItemType').text = 'Display'
+        etree.SubElement(item, 'Display').text = self.text
+        return item
+
+
+class Search:
+    def __init__(self, caption, url):
+        self.caption = caption
+        self.url = url
+
+    def append_to_xml(self, xml):
+        item = etree.SubElement(xml, 'Item')
+        etree.SubElement(item, 'ItemType').text = 'Search'
+        etree.SubElement(item, 'SearchURL').text = self.url
+        etree.SubElement(item, 'SearchURLBackUp').text = self.url
+        etree.SubElement(item, 'SearchCaption').text = self.caption
+        etree.SubElement(item, 'SearchTextbox').text = None
+        etree.SubElement(item, 'SearchButtonGo').text = "Search"
+        etree.SubElement(item, 'SearchButtonCancel').text = "Cancel"
+        return item
+
+
+class Directory:
+    def __init__(self, title, destination):
+        self.title = title
+        self.destination = destination
+
+    def append_to_xml(self, xml):
+        item = etree.SubElement(xml, 'Item')
+        etree.SubElement(item, 'ItemType').text = 'Dir'
+        etree.SubElement(item, 'Title').text = self.title
+        etree.SubElement(item, 'UrlDir').text = self.destination
+        etree.SubElement(item, 'UrlDirBackUp').text = self.destination
+        return item
+
+
+class Station:
+    def __init__(self, uid, name, description, url, logo, genre, location, mime, bitrate, bookmark):
+        self.uid = uid
+        self.name = name
+        self.description = description
+        self.url = url
+        self.logo = logo
+        self.genre = genre
+        self.location = location
+        self.mime = mime
+        self.bitrate = bitrate
+        self.bookmark = bookmark
+
+    def append_to_xml(self, xml):
+        item = etree.SubElement(xml, 'Item')
+        etree.SubElement(item, 'ItemType').text = 'Station'
+        etree.SubElement(item, 'StationId').text = self.uid
+        etree.SubElement(item, 'StationName').text = self.name
+        etree.SubElement(item, 'StationUrl').text = self.url
+        etree.SubElement(item, 'StationDesc').text = self.description
+        etree.SubElement(item, 'Logo').text = self.logo
+        etree.SubElement(item, 'StationFormat').text = self.genre
+        etree.SubElement(item, 'StationLocation').text = self.location
+        etree.SubElement(item, 'StationBandwidth').text = self.bitrate
+        etree.SubElement(item, 'StationMime').text = self.mime
+        etree.SubElement(item, 'StationBookmark').text = self.bookmark
+        return item