Merge pull request #77 from tomtastic/patch-1

Fix API endpoint for get_station_by_id

.gitignore

@@ -1,3 +1,6 @@
+build
+dist
+*.egg-info
 .idea
 *.iml
 *.pyc

LICENSE.txt

@@ -0,0 +1,14 @@
+Copyright (C) 2019 Micha LaQua
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.

MANIFEST.in

@@ -0,0 +1,2 @@
+include README.md
+include LICENCE.txt

README.md

@@ -2,81 +2,110 @@
 
 # YCast
 
+[![PyPI latest version](https://img.shields.io/pypi/v/ycast?color=success)](https://pypi.org/project/ycast/) [![GitHub latest version](https://img.shields.io/github/v/release/milaq/YCast?color=success&label=github&sort=semver)](https://github.com/milaq/YCast/releases) [![Python version](https://img.shields.io/pypi/pyversions/ycast)](https://www.python.org/downloads/) [![License](https://img.shields.io/pypi/l/ycast)](https://www.gnu.org/licenses/gpl-3.0.en.html) [![GitHub issues](https://img.shields.io/github/issues/milaq/ycast)](https://github.com/milaq/YCast/issues)
+
+[Get it via PyPI](https://pypi.org/project/ycast/)
+
+[Download from GitHub](https://github.com/milaq/YCast/releases)
+
+[Issue tracker](https://github.com/milaq/YCast/issues)
+
 YCast is a self hosted replacement for the vTuner internet radio service which many AVRs use.
-It emulates a vTuner backend to provide your AVR with the necessary information to play self defined categorized internet radio stations.
+It emulates a vTuner backend to provide your AVR with the necessary information to play self defined categorized internet radio stations and listen to Radio stations listed in the [Community Radio Browser index](http://www.radio-browser.info).
 
 YCast is for you if:
  * You do not want to use a proprietary streaming service
  * You are sick of loading delays and/or downtimes of the vTuner service
+ * You do not want to pay for a feature which was free before
  * You are unsure about the continuation of the vTuner service
 
 ## Supported devices
 
 Theoretically, YCast should work for **most AVRs which support vTuner**.
+Most AVRs with network connectivity that were produced between 2011 and 2017 have vTuner support built-in.
 
-Go ahead and test it with yours, and kindly report the result back :)
+Go ahead, test it with yours and kindly report the results back.
+Any reported device helps the community to see which AVRs work properly and which may have issues.
 
 ### Confirmed working
 
- * Yamaha RX-Vx73 series (RX-V373, RX-V473, RX-V573, RX-V673, RX-V773)
- * Yamaha R-N500
- * Onkyo TX-NR414
+ * Denon AVR-X_000 series (AVR-X1000, AVR-2000, AVR-X3000, AVR-X4000)
+ * Denon AVR-1912
+ * Denon AVR-X2200W
+ * Denon CEOL piccolo N5
+ * Denon CEOL N9
+ * Denon DNP-720AE
+ * Denon DNP-730AE
+ * Denon DRA-100
  * Marantz Melody Media M-CR610
+ * Marantz NR1506
+ * Marantz NR1605
+ * Marantz NA6005
+ * Marantz NA8005
+ * Marantz SR5009
+ * Onkyo TX-NR414
+ * Onkyo TX-NR5009
+ * Onkyo TX-NR616
+ * Yamaha R-N301
+ * Yamaha R-N500
+ * Yamaha RX-A810
+ * Yamaha RX-A820
+ * Yamaha RX-A830
+ * Yamaha CRX-N560/MCR-N560
+ * Yamaha RX-V_71 series with network connectivity (RX-V671, RX-V771)
+ * Yamaha RX-V_73 series with network connectivity (RX-V473, RX-V573, RX-V673, RX-V773)
+ * Yamaha RX-V_75 series (RX-V375, RX-V475, RX-V575, RX-V675, RX-V775)
+ * Yamaha RX-V_77 series (RX-V377, RX-V477, RX-V577, RX-V677, RX-V777)
+ * Yamaha RX-V3067
+ * Yamaha RX-V500D
 
 ### Unconfirmed/Experimental
 
- * Yamaha RX-Vx75 series (RX-V375, RX-V475, RX-V575, RX-V675, RX-V775)
- * Yamaha RX-Vx77 series (RX-V377, RX-V477, RX-V577, RX-V677, RX-V777)
- * Yamaha RX-Vx79 series (RX-V379, RX-V479, RX-V579, RX-V679, RX-V779)
- * Yamaha RX-Vx81 series (RX-V381, RX-V481, RX-V581, RX-V681, RX-V781)
+ * Denon AVR-X_100W series (AVR-X1100W, AVR-2100W, AVR-X3100W, AVR-X4100W)
+ * Denon AVR-X_300W series (AVR-X1300W, AVR-2300W, AVR-X3300W)
+ * Yamaha RX-A1060
+ * Yamaha CX-A5000
  * Yamaha RX-S600D
  * Yamaha RX-S601D
- * Yamaha WX-030
- * Yamaha RX-A1060
  * Yamaha RX-V2700
  * Yamaha RX-V3800
- * Yamaha CX-A5000
+ * Yamaha RX-V_79 series (RX-V379, RX-V479, RX-V579, RX-V679, RX-V779)
+ * Yamaha RX-V_81 series (RX-V381, RX-V481, RX-V581, RX-V681, RX-V781)
+ * Yamaha WX-030
 
 ## Dependencies:
 Python version: `3`
 
 Python packages:
+ * `requests`
+ * `flask`
  * `PyYAML`
+ * `Pillow`
  
 ## Usage
 
-YCast really does not need much computing power nor bandwidth. It just serves the information to the AVR. The streaming
-itself gets handled by the AVR directly, i.e. you can run it on a low-spec RISC machine like a Raspberry Pi.
+YCast really does not need much computing power nor bandwidth, i.e. you can run it on a low-spec RISC machine like a Raspberry Pi or a home router.
 
-1) Create your initial `stations.yml` and put it in the same directory as `ycast.py`. The config follows a basic YAML structure (see below).
-2) Create a manual entry in your DNS server (read 'Router' for most home users). `vtuner.com` should point to the machine YCast is running on. Alternatively, in case you only want to forward specific vendors, the following entries may be configured:
+### DNS entries
+
+You need to create a manual entry in your DNS server (read 'Router' for most home users). The `*.vtuner.com` domain should point to the machine YCast is running on.
+Specifically the following entries may be configured instead of a wildcard entry:
 
   * Yamaha AVRs: `radioyamaha.vtuner.com` (and optionally `radioyamaha2.vtuner.com`)
   * Onkyo AVRs: `onkyo.vtuner.com` (and optionally `onkyo2.vtuner.com`)
   * Denon/Marantz AVRs: `denon.vtuner.com` (and optionally `denon2.vtuner.com`)
-
-3) Run `ycast.py`.
-
-### stations.yml
-```
-Category one name:
-  First awesome station name: first.awesome/station/URL
-  Second awesome station name: second.awesome/station/URL
-
-Category two name:
-  Third awesome station name: third.awesome/station/URL
-  Fourth awesome station name: fourth.awesome/station/URL
-```   
-
-You can also have a look at the provided [example](examples/stations.yml.example) to better understand the configuration.
+  * Grundig radios: `grundig.vtuner.com`, `grundig.radiosetup.com` (and optionally `grundig2.vtuner.com` and `grundig2.radiosetup.com`)
 
 
-## Web server configuration
+### Running the server
+
+#### With built-in webserver
+
+You can run YCast by using the built-in development server of Flask (not recommended for production use, but should™ be enough for your private home use): `python -m ycast`
 
 While you can simply run YCast with root permissions listening on all interfaces on port 80, this may not be desired for various reasons.
 
-You can change the listen address and port (via `-l` and `-p` respectively) if you are already running a HTTP server on the target machine
-and/or want to proxy or restrict YCast access.
+You can change the listen address and port (via `-l` and `-p` respectively) if you are already running a HTTP server on the target machine and/or want to proxy or restrict YCast access.
 
 It is advised to use a proper webserver (e.g. Nginx) in front of YCast if you can.
 Then, you also don't need to run YCast as root and can proxy the requests to YCast running on a higher port (>1024) listening only on `localhost`.
@@ -85,21 +114,39 @@ You can redirect all traffic destined for the original request URL (e.g. `radioy
  * `/setupapp`
  * `/ycast`
 
-__Attention__: Do not rewrite the request transparently. YCast expects the complete URL (i.e. including `/ycast` or `/setupapp`). It also need an intact `Host` header; so if you're proxying YCast you need to pass the original header on. For Nginx, this can be accomplished with `proxy_set_header Host $host;`.
+__Attention__: Do not rewrite the requests transparently. YCast expects the complete URL (i.e. including `/ycast` or `/setupapp`). It also need an intact `Host` header; so if you're proxying YCast you need to pass the original header on. For Nginx, this can be accomplished with `proxy_set_header Host $host;`.
 
 In case you are using (or plan on using) Nginx to proxy requests, have a look at [this example](examples/nginx-ycast.conf.example).
 This can be used together with [this systemd service example](examples/ycast.service.example) for a fully functional deployment.
 
+#### With WSGI
+
+You can also setup a proper WSGI server. See the [official Flask documentation](https://flask.palletsprojects.com/en/1.1.x/deploying/).
+
+### Custom stations
+
+If you want to use the 'My Stations' feature, create a `stations.yml` and run YCast with the `-c` switch to specify the path to it. The config follows a basic YAML structure (see below).
+
+```
+Category one name:
+  First awesome station name: first.awesome/station/URL
+  Second awesome station name: second.awesome/station/URL
+
+Category two name:
+  Third awesome station name: third.awesome/station/URL
+  Fourth awesome station name: fourth.awesome/station/URL
+```
+
+You can also have a look at the provided [example](examples/stations.yml.example) to better understand the configuration.
+
 ## Firewall rules
 
- * The server running YCast does __not__ need internet access.
- * Your AVR needs access to the internet (i.e. to the station URLs you defined).
+ * Your AVR needs access to the internet.
  * Your AVR needs to reach port `80` of the machine running YCast.
+ * If you want to use Radiobrowser stations, the machine running YCast needs internet access.
 
 ## Caveats
 
-YCast was a quick and dirty project to lay the foundation for having a self hosted vTuner emulation.
-
-It is a barebone service at the moment. It provides your AVR with the basic info it needs to play internet radio stations. 
-Maybe this will change in the future, maybe not.
-For now just station names and URLs; no web-based management interface, no coverart, no cute kittens, no fancy stuff.
+ * vTuner compatible AVRs don't do HTTPS. As such, YCast blindly rewrites every HTTPS station URL to HTTP. Most station
+providers which utilize HTTPS for their stations also provide an HTTP stream. Thus, most HTTPS stations should work.
+ * The built-in bookmark function does not work at the moment. You need to manually add your favourite stations for now.

examples/nginx-ycast.conf.example

@@ -13,6 +13,6 @@ server {
                 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                 proxy_set_header X-Forwarded-Proto $scheme;
 
-                proxy_pass http://localhost:8010;
+                proxy_pass http://127.0.0.1:8010;
         }
 }

examples/ycast.service.example

@@ -6,7 +6,7 @@ After=network.target
 Type=simple
 User=ycast
 Group=ycast
-ExecStart=/opt/ycast/ycast.py -l 127.0.0.1 -p 8010
+ExecStart=/usr/bin/python3 -m ycast -l 127.0.0.1 -p 8010
 
 [Install]
 WantedBy=multi-user.target

setup.py

@@ -0,0 +1,45 @@
+from setuptools import setup, find_packages
+
+import ycast
+
+with open("README.md", "r") as fh:
+    long_description = fh.read()
+
+setup(
+    name='ycast',
+    version=ycast.__version__,
+    author='milaq',
+    author_email='micha.laqua@gmail.com',
+    description='Self hosted vTuner internet radio service emulation',
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+    url='https://github.com/milaq/YCast',
+    license='GPLv3',
+    classifiers=[
+        'Development Status :: 4 - Beta',
+        'Intended Audience :: Developers',
+        'Intended Audience :: System Administrators',
+        'License :: OSI Approved :: GNU General Public License v3 (GPLv3)',
+        'Natural Language :: English',
+        'Operating System :: POSIX :: Linux',
+        'Programming Language :: Python :: 3',
+        'Topic :: Multimedia :: Sound/Audio',
+        'Topic :: Software Development :: Libraries :: Python Modules'
+    ],
+    keywords=[
+        'ycast',
+        'streaming',
+        'vtuner',
+        'internet radio',
+        'music',
+        'radio',
+        'shoutcast',
+        'avr',
+        'emulation',
+        'yamaha',
+        'onkyo',
+        'denon'
+    ],
+    install_requires=['requests', 'flask', 'PyYAML', 'Pillow'],
+    packages=find_packages(exclude=['contrib', 'docs', 'tests'])
+)

ycast.py

@@ -1,105 +0,0 @@
-#!/usr/bin/env python3
-
-import os
-import sys
-import argparse
-from http.server import BaseHTTPRequestHandler, HTTPServer
-import xml.etree.cElementTree as etree
-
-import yaml
-
-YCAST_LOCATION = 'ycast'
-
-stations = {}
-
-
-def get_stations():
-    global stations
-    ycast_dir = os.path.dirname(os.path.realpath(__file__))
-    try:
-        with open(ycast_dir + '/stations.yml', 'r') as f:
-            stations = yaml.load(f)
-    except FileNotFoundError:
-        print("ERROR: Station configuration not found. Please supply a proper stations.yml.")
-        sys.exit(1)
-
-
-def text_to_url(text):
-    return text.replace(' ', '%20')
-
-
-def url_to_text(url):
-    return url.replace('%20', ' ')
-
-
-class YCastServer(BaseHTTPRequestHandler):
-    def do_GET(self):
-        get_stations()
-        self.address = 'http://' + self.headers['Host']
-        if 'loginXML.asp?token=0' in self.path:
-            self.send_xml('<EncryptedToken>0000000000000000</EncryptedToken>')
-        elif self.path == '/' \
-                or self.path == '/' + YCAST_LOCATION \
-                or self.path == '/' + YCAST_LOCATION + '/'\
-                or self.path.startswith('/setupapp'):
-            xml = self.create_root()
-            for category in sorted(stations, key=str.lower):
-                self.add_dir(xml, category,
-                             self.address + '/' + YCAST_LOCATION + '/' + text_to_url(category))
-            self.send_xml(etree.tostring(xml).decode('utf-8'))
-        elif self.path.startswith('/' + YCAST_LOCATION + '/'):
-            category = url_to_text(self.path[len(YCAST_LOCATION) + 2:].partition('?')[0])
-            if category not in stations:
-                self.send_error(404)
-                return
-            xml = self.create_root()
-            for station in sorted(stations[category], key=str.lower):
-                self.add_station(xml, station, stations[category][station])
-            self.send_xml(etree.tostring(xml).decode('utf-8'))
-        else:
-            self.send_error(404)
-
-    def send_xml(self, content):
-        xml_data = '<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>'
-        xml_data += content
-        self.send_response(200)
-        self.send_header('Content-type', 'text/html')
-        self.send_header('Content-length', len(xml_data))
-        self.end_headers()
-        self.wfile.write(bytes(xml_data, 'utf-8'))
-
-    def create_root(self):
-        return etree.Element('ListOfItems')
-
-    def add_dir(self, root, name, dest):
-        item = etree.SubElement(root, 'Item')
-        etree.SubElement(item, 'ItemType').text = 'Dir'
-        etree.SubElement(item, 'Title').text = name
-        etree.SubElement(item, 'UrlDir').text = dest
-        return item
-
-    def add_station(self, root, name, url):
-        item = etree.SubElement(root, 'Item')
-        etree.SubElement(item, 'ItemType').text = 'Station'
-        etree.SubElement(item, 'StationName').text = name
-        etree.SubElement(item, 'StationUrl').text = url
-        return item
-
-
-parser = argparse.ArgumentParser(description='vTuner API emulation')
-parser.add_argument('-l', action='store', dest='address', help='Listen address', default='0.0.0.0')
-parser.add_argument('-p', action='store', dest='port', type=int, help='Listen port', default=80)
-arguments = parser.parse_args()
-get_stations()
-try:
-    server = HTTPServer((arguments.address, arguments.port), YCastServer)
-except PermissionError:
-    print("ERROR: No permission to create socket. Are you trying to use ports below 1024 without elevated rights?")
-    sys.exit(1)
-print('YCast server listening on %s:%s' % (arguments.address, arguments.port))
-try:
-    server.serve_forever()
-except KeyboardInterrupt:
-    pass
-print('YCast server shutting down')
-server.server_close()

ycast/__init__.py

@@ -0,0 +1 @@
+__version__ = '1.1.0'

ycast/__main__.py

@@ -0,0 +1,34 @@
+#!/usr/bin/env python3
+
+import argparse
+import logging
+import sys
+
+from ycast import __version__
+from ycast import server
+
+logging.basicConfig(format='%(asctime)s %(levelname)s: %(message)s', datefmt='%Y-%m-%d %H:%M:%S', level=logging.INFO)
+
+
+def launch_server():
+    parser = argparse.ArgumentParser(description='vTuner API emulation')
+    parser.add_argument('-c', action='store', dest='config', help='Station configuration', default=None)
+    parser.add_argument('-l', action='store', dest='address', help='Listen address', default='0.0.0.0')
+    parser.add_argument('-p', action='store', dest='port', type=int, help='Listen port', default=80)
+    parser.add_argument('-d', action='store_true', dest='debug', help='Enable debug logging')
+    arguments = parser.parse_args()
+    logging.info("YCast (%s) server starting", __version__)
+    if arguments.debug:
+        logging.getLogger().setLevel(logging.DEBUG)
+        logging.debug("Debug logging enabled")
+    else:
+        logging.getLogger('werkzeug').setLevel(logging.WARNING)
+    server.run(arguments.config, arguments.address, arguments.port)
+
+
+if __name__ == "__main__":
+    if sys.version_info[0] < 3:
+        logging.error("Unsupported Python version (Python %s). Minimum required version is Python 3.",
+                      sys.version_info[0])
+        sys.exit(1)
+    launch_server()

ycast/generic.py

@@ -0,0 +1,52 @@
+import logging
+import os
+
+USER_AGENT = 'YCast'
+VAR_PATH = os.path.expanduser("~") + '/.ycast'
+CACHE_PATH = VAR_PATH + '/cache'
+
+
+class Directory:
+    def __init__(self, name, item_count, displayname=None):
+        self.name = name
+        self.item_count = item_count
+        if displayname:
+            self.displayname = displayname
+        else:
+            self.displayname = name
+
+
+def generate_stationid_with_prefix(uid, prefix):
+    if not prefix or len(prefix) != 2:
+        logging.error("Invalid station prefix length (must be 2)")
+        return None
+    if not uid:
+        logging.error("Missing station id for full station id generation")
+        return None
+    return str(prefix) + '_' + str(uid)
+
+
+def get_stationid_prefix(uid):
+    if len(uid) < 4:
+        logging.error("Could not extract stationid (Invalid station id length)")
+        return None
+    return uid[:2]
+
+
+def get_stationid_without_prefix(uid):
+    if len(uid) < 4:
+        logging.error("Could not extract stationid (Invalid station id length)")
+        return None
+    return uid[3:]
+
+
+def get_cache_path(cache_name):
+    cache_path = CACHE_PATH + '/' + cache_name
+    try:
+        os.makedirs(cache_path)
+    except FileExistsError:
+        pass
+    except PermissionError:
+        logging.error("Could not create cache folders (%s) because of access permissions", cache_path)
+        return None
+    return cache_path

ycast/my_stations.py

@@ -0,0 +1,87 @@
+import logging
+import hashlib
+
+import yaml
+
+import ycast.vtuner as vtuner
+import ycast.generic as generic
+
+ID_PREFIX = "MY"
+
+config_file = 'stations.yml'
+
+
+class Station:
+    def __init__(self, uid, name, url, category):
+        self.id = generic.generate_stationid_with_prefix(uid, ID_PREFIX)
+        self.name = name
+        self.url = url
+        self.tag = category
+        self.icon = None
+
+    def to_vtuner(self):
+        return vtuner.Station(self.id, self.name, self.tag, self.url, self.icon, self.tag, None, None, None, None)
+
+
+def set_config(config):
+    global config_file
+    if config:
+        config_file = config
+    if get_stations_yaml():
+        return True
+    else:
+        return False
+
+
+def get_station_by_id(uid):
+    my_stations_yaml = get_stations_yaml()
+    if my_stations_yaml:
+        for category in my_stations_yaml:
+            for station in get_stations_by_category(category):
+                if uid == generic.get_stationid_without_prefix(station.id):
+                    return station
+    return None
+
+
+def get_stations_yaml():
+    try:
+        with open(config_file, 'r') as f:
+            my_stations = yaml.safe_load(f)
+    except FileNotFoundError:
+        logging.error("Station configuration '%s' not found", config_file)
+        return None
+    except yaml.YAMLError as e:
+        logging.error("Station configuration format error: %s", e)
+        return None
+    return my_stations
+
+
+def get_category_directories():
+    my_stations_yaml = get_stations_yaml()
+    categories = []
+    if my_stations_yaml:
+        for category in my_stations_yaml:
+            categories.append(generic.Directory(category, len(get_stations_by_category(category))))
+    return categories
+
+
+def get_stations_by_category(category):
+    my_stations_yaml = get_stations_yaml()
+    stations = []
+    if my_stations_yaml and category in my_stations_yaml:
+        for station_name in my_stations_yaml[category]:
+            station_url = my_stations_yaml[category][station_name]
+            station_id = str(get_checksum(station_name + station_url)).upper()
+            stations.append(Station(station_id, station_name, station_url, category))
+    return stations
+
+
+def get_checksum(feed, charlimit=12):
+    hash_feed = feed.encode()
+    hash_object = hashlib.md5(hash_feed)
+    digest = hash_object.digest()
+    xor_fold = bytearray(digest[:8])
+    for i, b in enumerate(digest[8:]):
+        xor_fold[i] ^= b
+    digest_xor_fold = ''.join(format(x, '02x') for x in bytes(xor_fold))
+    return digest_xor_fold[:charlimit]

ycast/radiobrowser.py

@@ -0,0 +1,157 @@
+import requests
+import logging
+
+from ycast import __version__
+import ycast.vtuner as vtuner
+import ycast.generic as generic
+
+API_ENDPOINT = "http://all.api.radio-browser.info"
+MINIMUM_COUNT_GENRE = 5
+MINIMUM_COUNT_COUNTRY = 5
+MINIMUM_COUNT_LANGUAGE = 5
+DEFAULT_STATION_LIMIT = 200
+SHOW_BROKEN_STATIONS = False
+ID_PREFIX = "RB"
+
+
+def get_json_attr(json, attr):
+    try:
+        return json[attr]
+    except:
+        return None
+
+
+class Station:
+    def __init__(self, station_json):
+        self.id = generic.generate_stationid_with_prefix(get_json_attr(station_json, 'stationuuid'), ID_PREFIX)
+        self.name = get_json_attr(station_json, 'name')
+        self.url = get_json_attr(station_json, 'url')
+        self.icon = get_json_attr(station_json, 'favicon')
+        self.tags = get_json_attr(station_json, 'tags').split(',')
+        self.countrycode = get_json_attr(station_json, 'countrycode')
+        self.language = get_json_attr(station_json, 'language')
+        self.votes = get_json_attr(station_json, 'votes')
+        self.codec = get_json_attr(station_json, 'codec')
+        self.bitrate = get_json_attr(station_json, 'bitrate')
+
+    def to_vtuner(self):
+        return vtuner.Station(self.id, self.name, ', '.join(self.tags), self.url, self.icon,
+                              self.tags[0], self.countrycode, self.codec, self.bitrate, None)
+
+    def get_playable_url(self):
+        try:
+            playable_url_json = request('url/' + generic.get_stationid_without_prefix(self.id))[0]
+            self.url = playable_url_json['url']
+        except (IndexError, KeyError):
+            logging.error("Could not retrieve first playlist item for station with id '%s'", self.id)
+
+
+def request(url):
+    logging.debug("Radiobrowser API request: %s", url)
+    headers = {'content-type': 'application/json', 'User-Agent': generic.USER_AGENT + '/' + __version__}
+    try:
+        response = requests.get(API_ENDPOINT + '/json/' + url, headers=headers)
+    except requests.exceptions.ConnectionError as err:
+        logging.error("Connection to Radiobrowser API failed (%s)", err)
+        return {}
+    if response.status_code != 200:
+        logging.error("Could not fetch data from Radiobrowser API (HTML status %s)", response.status_code)
+        return {}
+    return response.json()
+
+
+def get_station_by_id(uid):
+    station_json = request('stations/byuuid/' + str(uid))
+    if station_json and len(station_json):
+        return Station(station_json[0])
+    else:
+        return None
+
+
+def search(name, limit=DEFAULT_STATION_LIMIT):
+    stations = []
+    stations_json = request('stations/search?order=name&reverse=false&limit=' + str(limit) + '&name=' + str(name))
+    for station_json in stations_json:
+        if SHOW_BROKEN_STATIONS or get_json_attr(station_json, 'lastcheckok') == 1:
+            stations.append(Station(station_json))
+    return stations
+
+
+def get_country_directories():
+    country_directories = []
+    apicall = 'countries'
+    if not SHOW_BROKEN_STATIONS:
+        apicall += '?hidebroken=true'
+    countries_raw = request(apicall)
+    for country_raw in countries_raw:
+        if get_json_attr(country_raw, 'name') and get_json_attr(country_raw, 'stationcount') and \
+                int(get_json_attr(country_raw, 'stationcount')) > MINIMUM_COUNT_COUNTRY:
+            country_directories.append(generic.Directory(get_json_attr(country_raw, 'name'),
+                                                         get_json_attr(country_raw, 'stationcount')))
+    return country_directories
+
+
+def get_language_directories():
+    language_directories = []
+    apicall = 'languages'
+    if not SHOW_BROKEN_STATIONS:
+        apicall += '?hidebroken=true'
+    languages_raw = request(apicall)
+    for language_raw in languages_raw:
+        if get_json_attr(language_raw, 'name') and get_json_attr(language_raw, 'stationcount') and \
+                int(get_json_attr(language_raw, 'stationcount')) > MINIMUM_COUNT_LANGUAGE:
+            language_directories.append(generic.Directory(get_json_attr(language_raw, 'name'),
+                                                          get_json_attr(language_raw, 'stationcount'),
+                                                          get_json_attr(language_raw, 'name').title()))
+    return language_directories
+
+
+def get_genre_directories():
+    genre_directories = []
+    apicall = 'tags'
+    if not SHOW_BROKEN_STATIONS:
+        apicall += '?hidebroken=true'
+    genres_raw = request(apicall)
+    for genre_raw in genres_raw:
+        if get_json_attr(genre_raw, 'name') and get_json_attr(genre_raw, 'stationcount') and \
+                int(get_json_attr(genre_raw, 'stationcount')) > MINIMUM_COUNT_GENRE:
+            genre_directories.append(generic.Directory(get_json_attr(genre_raw, 'name'),
+                                                       get_json_attr(genre_raw, 'stationcount'),
+                                                       get_json_attr(genre_raw, 'name').capitalize()))
+    return genre_directories
+
+
+def get_stations_by_country(country):
+    stations = []
+    stations_json = request('stations/search?order=name&reverse=false&countryExact=true&country=' + str(country))
+    for station_json in stations_json:
+        if SHOW_BROKEN_STATIONS or get_json_attr(station_json, 'lastcheckok') == 1:
+            stations.append(Station(station_json))
+    return stations
+
+
+def get_stations_by_language(language):
+    stations = []
+    stations_json = request('stations/search?order=name&reverse=false&languageExact=true&language=' + str(language))
+    for station_json in stations_json:
+        if SHOW_BROKEN_STATIONS or get_json_attr(station_json, 'lastcheckok') == 1:
+            stations.append(Station(station_json))
+    return stations
+
+
+def get_stations_by_genre(genre):
+    stations = []
+    stations_json = request('stations/search?order=name&reverse=false&tagExact=true&tag=' + str(genre))
+    for station_json in stations_json:
+        if SHOW_BROKEN_STATIONS or get_json_attr(station_json, 'lastcheckok') == 1:
+            stations.append(Station(station_json))
+    return stations
+
+
+def get_stations_by_votes(limit=DEFAULT_STATION_LIMIT):
+    stations = []
+    stations_json = request('stations?order=votes&reverse=true&limit=' + str(limit))
+    for station_json in stations_json:
+        if SHOW_BROKEN_STATIONS or get_json_attr(station_json, 'lastcheckok') == 1:
+            stations.append(Station(station_json))
+    return stations

ycast/server.py

@@ -0,0 +1,311 @@
+import logging
+import re
+
+from flask import Flask, request, url_for, redirect, abort, make_response
+
+import ycast.vtuner as vtuner
+import ycast.radiobrowser as radiobrowser
+import ycast.my_stations as my_stations
+import ycast.generic as generic
+import ycast.station_icons as station_icons
+
+
+PATH_ROOT = 'ycast'
+PATH_PLAY = 'play'
+PATH_STATION = 'station'
+PATH_SEARCH = 'search'
+PATH_ICON = 'icon'
+PATH_MY_STATIONS = 'my_stations'
+PATH_RADIOBROWSER = 'radiobrowser'
+PATH_RADIOBROWSER_COUNTRY = 'country'
+PATH_RADIOBROWSER_LANGUAGE = 'language'
+PATH_RADIOBROWSER_GENRE = 'genre'
+PATH_RADIOBROWSER_POPULAR = 'popular'
+
+station_tracking = False
+my_stations_enabled = False
+app = Flask(__name__)
+
+
+def run(config, address='0.0.0.0', port=8010):
+    try:
+        check_my_stations_feature(config)
+        app.run(host=address, port=port)
+    except PermissionError:
+        logging.error("No permission to create socket. Are you trying to use ports below 1024 without elevated rights?")
+
+
+def check_my_stations_feature(config):
+    global my_stations_enabled
+    my_stations_enabled = my_stations.set_config(config)
+
+
+def get_directories_page(subdir, directories, request):
+    page = vtuner.Page()
+    if len(directories) == 0:
+        page.add(vtuner.Display("No entries found"))
+        page.set_count(1)
+        return page
+    for directory in get_paged_elements(directories, request.args):
+        vtuner_directory = vtuner.Directory(directory.displayname,
+                                            url_for(subdir, _external=True, directory=directory.name),
+                                            directory.item_count)
+        page.add(vtuner_directory)
+    page.set_count(len(directories))
+    return page
+
+
+def get_stations_page(stations, request):
+    page = vtuner.Page()
+    if len(stations) == 0:
+        page.add(vtuner.Display("No stations found"))
+        page.set_count(1)
+        return page
+    for station in get_paged_elements(stations, request.args):
+        vtuner_station = station.to_vtuner()
+        if station_tracking:
+            vtuner_station.set_trackurl(request.host_url + PATH_ROOT + '/' + PATH_PLAY + '?id=' + vtuner_station.uid)
+        vtuner_station.icon = request.host_url + PATH_ROOT + '/' + PATH_ICON + '?id=' + vtuner_station.uid
+        page.add(vtuner_station)
+    page.set_count(len(stations))
+    return page
+
+
+def get_paged_elements(items, requestargs):
+    if requestargs.get('startitems'):
+        offset = int(requestargs.get('startitems')) - 1
+    elif requestargs.get('startItems'):
+        offset = int(requestargs.get('startItems')) - 1
+    elif requestargs.get('start'):
+        offset = int(requestargs.get('start')) - 1
+    else:
+        offset = 0
+    if offset > len(items):
+        logging.warning("Paging offset larger than item count")
+        return []
+    if requestargs.get('enditems'):
+        limit = int(requestargs.get('enditems'))
+    elif requestargs.get('endItems'):
+        limit = int(requestargs.get('endItems'))
+    elif requestargs.get('start') and requestargs.get('howmany'):
+        limit = int(requestargs.get('start')) - 1 + int(requestargs.get('howmany'))
+    else:
+        limit = len(items)
+    if limit < offset:
+        logging.warning("Paging limit smaller than offset")
+        return []
+    if limit > len(items):
+        limit = len(items)
+    return items[offset:limit]
+
+
+def get_station_by_id(stationid, additional_info=False):
+    station_id_prefix = generic.get_stationid_prefix(stationid)
+    if station_id_prefix == my_stations.ID_PREFIX:
+        return my_stations.get_station_by_id(generic.get_stationid_without_prefix(stationid))
+    elif station_id_prefix == radiobrowser.ID_PREFIX:
+        station = radiobrowser.get_station_by_id(generic.get_stationid_without_prefix(stationid))
+        if additional_info:
+            station.get_playable_url()
+        return station
+    return None
+
+
+def vtuner_redirect(url):
+    if request and request.host and not re.search("^[A-Za-z0-9]+\.vtuner\.com$", request.host):
+        logging.warning("You are not accessing a YCast redirect with a whitelisted host url (*.vtuner.com). "
+                        "Some AVRs have problems with this. The requested host was: %s", request.host)
+    return redirect(url, code=302)
+
+
+@app.route('/setupapp/<path:path>',
+           methods=['GET', 'POST'])
+def upstream(path):
+    if request.args.get('token') == '0':
+        return vtuner.get_init_token()
+    if request.args.get('search'):
+        return station_search()
+    if 'statxml.asp' in path and request.args.get('id'):
+        return get_station_info()
+    if 'navXML.asp' in path:
+        return radiobrowser_landing()
+    if 'FavXML.asp' in path:
+        return my_stations_landing()
+    if 'loginXML.asp' in path:
+        return landing()
+    logging.error("Unhandled upstream query (/setupapp/%s)", path)
+    abort(404)
+
+
+@app.route('/',
+           defaults={'path': ''},
+           methods=['GET', 'POST'])
+@app.route('/' + PATH_ROOT + '/',
+           defaults={'path': ''},
+           methods=['GET', 'POST'])
+def landing(path=''):
+    page = vtuner.Page()
+    page.add(vtuner.Directory('Radiobrowser', url_for('radiobrowser_landing', _external=True), 4))
+    if my_stations_enabled:
+        page.add(vtuner.Directory('My Stations', url_for('my_stations_landing', _external=True),
+                                  len(my_stations.get_category_directories())))
+    else:
+        page.add(vtuner.Display("'My Stations' feature not configured."))
+        page.set_count(1)
+    return page.to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_MY_STATIONS + '/',
+           methods=['GET', 'POST'])
+def my_stations_landing():
+    directories = my_stations.get_category_directories()
+    return get_directories_page('my_stations_category', directories, request).to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_MY_STATIONS + '/<directory>',
+           methods=['GET', 'POST'])
+def my_stations_category(directory):
+    stations = my_stations.get_stations_by_category(directory)
+    return get_stations_page(stations, request).to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/',
+           methods=['GET', 'POST'])
+def radiobrowser_landing():
+    page = vtuner.Page()
+    page.add(vtuner.Directory('Genres', url_for('radiobrowser_genres', _external=True),
+                              len(radiobrowser.get_genre_directories())))
+    page.add(vtuner.Directory('Countries', url_for('radiobrowser_countries', _external=True),
+                              len(radiobrowser.get_country_directories())))
+    page.add(vtuner.Directory('Languages', url_for('radiobrowser_languages', _external=True),
+                              len(radiobrowser.get_language_directories())))
+    page.add(vtuner.Directory('Most Popular', url_for('radiobrowser_popular', _external=True),
+                              len(radiobrowser.get_stations_by_votes())))
+    page.set_count(4)
+    return page.to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_COUNTRY + '/',
+           methods=['GET', 'POST'])
+def radiobrowser_countries():
+    directories = radiobrowser.get_country_directories()
+    return get_directories_page('radiobrowser_country_stations', directories, request).to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_COUNTRY + '/<directory>',
+           methods=['GET', 'POST'])
+def radiobrowser_country_stations(directory):
+    stations = radiobrowser.get_stations_by_country(directory)
+    return get_stations_page(stations, request).to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_LANGUAGE + '/',
+           methods=['GET', 'POST'])
+def radiobrowser_languages():
+    directories = radiobrowser.get_language_directories()
+    return get_directories_page('radiobrowser_language_stations', directories, request).to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_LANGUAGE + '/<directory>',
+           methods=['GET', 'POST'])
+def radiobrowser_language_stations(directory):
+    stations = radiobrowser.get_stations_by_language(directory)
+    return get_stations_page(stations, request).to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_GENRE + '/',
+           methods=['GET', 'POST'])
+def radiobrowser_genres():
+    directories = radiobrowser.get_genre_directories()
+    return get_directories_page('radiobrowser_genre_stations', directories, request).to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_GENRE + '/<directory>',
+           methods=['GET', 'POST'])
+def radiobrowser_genre_stations(directory):
+    stations = radiobrowser.get_stations_by_genre(directory)
+    return get_stations_page(stations, request).to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_RADIOBROWSER + '/' + PATH_RADIOBROWSER_POPULAR + '/',
+           methods=['GET', 'POST'])
+def radiobrowser_popular():
+    stations = radiobrowser.get_stations_by_votes()
+    return get_stations_page(stations, request).to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_SEARCH + '/',
+           methods=['GET', 'POST'])
+def station_search():
+    query = request.args.get('search')
+    if not query or len(query) < 3:
+        page = vtuner.Page()
+        page.add(vtuner.Display("Search query too short"))
+        page.set_count(1)
+        return page.to_string()
+    else:
+        # TODO: we also need to include 'my station' elements
+        stations = radiobrowser.search(query)
+        return get_stations_page(stations, request).to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_PLAY,
+           methods=['GET', 'POST'])
+def get_stream_url():
+    stationid = request.args.get('id')
+    if not stationid:
+        logging.error("Stream URL without station ID requested")
+        abort(400)
+    station = get_station_by_id(stationid, additional_info=True)
+    if not station:
+        logging.error("Could not get station with id '%s'", stationid)
+        abort(404)
+    logging.debug("Station with ID '%s' requested", station.id)
+    return vtuner_redirect(station.url)
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_STATION,
+           methods=['GET', 'POST'])
+def get_station_info():
+    stationid = request.args.get('id')
+    if not stationid:
+        logging.error("Station info without station ID requested")
+        abort(400)
+    station = get_station_by_id(stationid, additional_info=(not station_tracking))
+    if not station:
+        logging.error("Could not get station with id '%s'", stationid)
+        page = vtuner.Page()
+        page.add(vtuner.Display("Station not found"))
+        page.set_count(1)
+        return page.to_string()
+    vtuner_station = station.to_vtuner()
+    if station_tracking:
+        vtuner_station.set_trackurl(request.host_url + PATH_ROOT + '/' + PATH_PLAY + '?id=' + vtuner_station.uid)
+    vtuner_station.icon = request.host_url + PATH_ROOT + '/' + PATH_ICON + '?id=' + vtuner_station.uid
+    page = vtuner.Page()
+    page.add(vtuner_station)
+    page.set_count(1)
+    return page.to_string()
+
+
+@app.route('/' + PATH_ROOT + '/' + PATH_ICON,
+           methods=['GET', 'POST'])
+def get_station_icon():
+    stationid = request.args.get('id')
+    if not stationid:
+        logging.error("Station icon without station ID requested")
+        abort(400)
+    station = get_station_by_id(stationid)
+    if not station:
+        logging.error("Could not get station with id '%s'", stationid)
+        abort(404)
+    if not hasattr(station, 'icon') or not station.icon:
+        logging.warning("No icon information found for station with id '%s'", stationid)
+        abort(404)
+    station_icon = station_icons.get_icon(station)
+    if not station_icon:
+        logging.error("Could not get station icon for station with id '%s'", stationid)
+        abort(404)
+    response = make_response(station_icon)
+    response.headers.set('Content-Type', 'image/jpeg')
+    return response

ycast/station_icons.py

@@ -0,0 +1,50 @@
+import logging
+import requests
+import io
+import os
+
+from PIL import Image
+
+import ycast.generic as generic
+from ycast import __version__
+
+MAX_SIZE = 290
+CACHE_NAME = 'icons'
+
+
+def get_icon(station):
+    cache_path = generic.get_cache_path(CACHE_NAME)
+    if not cache_path:
+        return None
+    station_icon_file = cache_path + '/' + station.id
+    if not os.path.exists(station_icon_file):
+        logging.debug("Station icon cache miss. Fetching and converting station icon for station id '%s'", station.id)
+        headers = {'User-Agent': generic.USER_AGENT + '/' + __version__}
+        try:
+            response = requests.get(station.icon, headers=headers)
+        except requests.exceptions.ConnectionError as err:
+            logging.error("Connection to station icon URL failed (%s)", err)
+            return None
+        if response.status_code != 200:
+            logging.error("Could not get station icon data from %s (HTML status %s)", station.icon, response.status_code)
+            return None
+        try:
+            image = Image.open(io.BytesIO(response.content))
+            image = image.convert("RGB")
+            if image.size[0] > image.size[1]:
+                ratio = MAX_SIZE / image.size[0]
+            else:
+                ratio = MAX_SIZE / image.size[1]
+            image = image.resize((int(image.size[0] * ratio), int(image.size[1] * ratio)), Image.ANTIALIAS)
+            image.save(station_icon_file, format="JPEG")
+        except Exception as e:
+            logging.error("Station icon conversion error (%s)", e)
+            return None
+    try:
+        with open(station_icon_file, 'rb') as file:
+            image_conv = file.read()
+    except PermissionError:
+        logging.error("Could not access station icon file in cache (%s) because of access permissions",
+                      station_icon_file)
+        return None
+    return image_conv

ycast/vtuner.py

@@ -0,0 +1,144 @@
+import xml.etree.ElementTree as ET
+
+XML_HEADER = '<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>'
+
+
+def get_init_token():
+    return '<EncryptedToken>0000000000000000</EncryptedToken>'
+
+
+def strip_https(url):
+    if url.startswith('https://'):
+        url = 'http://' + url[8:]
+    return url
+
+
+def add_bogus_parameter(url):
+    """
+    We need this bogus parameter because some (if not all) AVRs blindly append additional request parameters
+    with an ampersand. E.g.: '&mac=<REDACTED>&dlang=eng&fver=1.2&startitems=1&enditems=100'.
+    The original vTuner API hacks around that by adding a specific parameter or a bogus parameter like '?empty=' to
+    the target URL.
+    """
+    return url + '?vtuner=true'
+
+
+class Page:
+    def __init__(self):
+        self.items = []
+        self.count = -1
+        self.dontcache = False
+
+    def add(self, item):
+        self.items.append(item)
+
+    def set_count(self, count):
+        self.count = count
+
+    def to_xml(self):
+        xml = ET.Element('ListOfItems')
+        ET.SubElement(xml, 'ItemCount').text = str(self.count)
+        if self.dontcache:
+            ET.SubElement(xml, 'NoDataCache').text = 'Yes'
+        for item in self.items:
+            xml.append(item.to_xml())
+        return xml
+
+    def to_string(self):
+        return XML_HEADER + ET.tostring(self.to_xml()).decode('utf-8')
+
+
+class Previous:
+    def __init__(self, url):
+        self.url = url
+
+    def to_xml(self):
+        item = ET.Element('Item')
+        ET.SubElement(item, 'ItemType').text = 'Previous'
+        ET.SubElement(item, 'UrlPrevious').text = add_bogus_parameter(self.url)
+        ET.SubElement(item, 'UrlPreviousBackUp').text = add_bogus_parameter(self.url)
+        return item
+
+
+class Display:
+    def __init__(self, text):
+        self.text = text
+
+    def to_xml(self):
+        item = ET.Element('Item')
+        ET.SubElement(item, 'ItemType').text = 'Display'
+        ET.SubElement(item, 'Display').text = self.text
+        return item
+
+
+class Search:
+    def __init__(self, caption, url):
+        self.caption = caption
+        self.url = url
+
+    def to_xml(self):
+        item = ET.Element('Item')
+        ET.SubElement(item, 'ItemType').text = 'Search'
+        ET.SubElement(item, 'SearchURL').text = add_bogus_parameter(self.url)
+        ET.SubElement(item, 'SearchURLBackUp').text = add_bogus_parameter(self.url)
+        ET.SubElement(item, 'SearchCaption').text = self.caption
+        ET.SubElement(item, 'SearchTextbox').text = None
+        ET.SubElement(item, 'SearchButtonGo').text = "Search"
+        ET.SubElement(item, 'SearchButtonCancel').text = "Cancel"
+        return item
+
+
+class Directory:
+    def __init__(self, title, destination, item_count=-1):
+        self.title = title
+        self.destination = destination
+        self.item_count = item_count
+
+    def to_xml(self):
+        item = ET.Element('Item')
+        ET.SubElement(item, 'ItemType').text = 'Dir'
+        ET.SubElement(item, 'Title').text = self.title
+        ET.SubElement(item, 'UrlDir').text = add_bogus_parameter(self.destination)
+        ET.SubElement(item, 'UrlDirBackUp').text = add_bogus_parameter(self.destination)
+        ET.SubElement(item, 'DirCount').text = str(self.item_count)
+        return item
+
+    def set_item_count(self, item_count):
+        self.item_count = item_count
+
+
+class Station:
+    def __init__(self, uid, name, description, url, icon, genre, location, mime, bitrate, bookmark):
+        self.uid = uid
+        self.name = name
+        self.description = description
+        self.url = strip_https(url)
+        self.trackurl = None
+        self.icon = icon
+        self.genre = genre
+        self.location = location
+        self.mime = mime
+        self.bitrate = bitrate
+        self.bookmark = bookmark
+
+    def set_trackurl(self, url):
+        self.trackurl = url
+
+    def to_xml(self):
+        item = ET.Element('Item')
+        ET.SubElement(item, 'ItemType').text = 'Station'
+        ET.SubElement(item, 'StationId').text = self.uid
+        ET.SubElement(item, 'StationName').text = self.name
+        if self.trackurl:
+            ET.SubElement(item, 'StationUrl').text = self.trackurl
+        else:
+            ET.SubElement(item, 'StationUrl').text = self.url
+        ET.SubElement(item, 'StationDesc').text = self.description
+        ET.SubElement(item, 'Logo').text = self.icon
+        ET.SubElement(item, 'StationFormat').text = self.genre
+        ET.SubElement(item, 'StationLocation').text = self.location
+        ET.SubElement(item, 'StationBandWidth').text = str(self.bitrate)
+        ET.SubElement(item, 'StationMime').text = self.mime
+        ET.SubElement(item, 'Relia').text = '3'
+        ET.SubElement(item, 'Bookmark').text = self.bookmark
+        return item