beenull/YCast
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Merge pull request #3 from superclass/master

Browse source 
Overhaul of filter/limits
This commit is contained in:
Thomas Hanika 2023-02-24 11:12:34 +01:00 committed by GitHub
commit cabcb713fb
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
7 changed files with 168 additions and 33 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
README.md
View file

@ -149,16 +149,16 @@ Category two name:
You can also have a look at the provided [example](examples/stations.yml.example) to better understand the configuration.
### Filter/limits
The filter configuration file .ycast/filter.yml (see example) allows to filter stations based on a whitelist / blacklist. The contents of this list specifies which attributes to filter on.
As the amount of stations can be overwhelming on a AV receiver interface Ycast allows for filtering. The filter configuration file .ycast/filter.yml allows to filter stations based on a whitelist / blacklist. The contents of this list specifies which attributes to filter on. Look at the provided [example](examples/filter.yml.example) for the details.
The limits allow to filter out genres, countries and languages that fail to have a certain amount of items. It also sets the default station limit and allows to show or hide broken stations. Defaults are as follows:
The limits allow to filter out genres, countries and languages that fail to have a certain amount of items. It also sets the default station limit for search and votes and allows to show or hide broken stations. Defaults are as follows:
* MINIMUM_COUNT_GENRE : 40
* MINIMUM_COUNT_COUNTRY : 5
* MINIMUM_COUNT_LANGUAGE : 5
* DEFAULT_STATION_LIMIT : 200
* SHOW_BROKEN_STATIONS : False
You can set your own valies in .ycast/filter.xml by adding these attributes and there values in the limits list. The filter file is not reread automatically when modified while the server is running. Send a HUP signal to trigger but it's preferred to use the api (see below) to modify the lists.
You can set your own values in filter.xml by adding these attributes and values in the limits list. The filter file is not reread automatically when modified while the server is running. Send a HUP signal to trigger but it's preferred to use the api (see below) to modify the lists.
The current filters/limits can be queried through a REST API by calling the GET method on /control/filter/whitelist, /control/filter/blacklist and /control/filter/limits. They can be modified by using the POST method an posting a JSON with the items to modify. Specifying a null value for an item will delete it from the list or, in the case of the limits, reset it to its default.

24
examples/apicalls.sh Normal file
View file

@ -0,0 +1,24 @@
ENDPOINT=127.0.0.1
# API
# Get recently played stations
curl "http://${ENDPOINT}/api/stations?category=recently"
# Get highest rated stations
curl "http://${ENDPOINT}/api/stations?category=voted"
# Get stations by language, specify by language paramter (default=german)
curl "http://${ENDPOINT}/api/stations?category=language&language=dutch"
# Get stations by country, specify by country paramter (default=Germany)
curl "http://${ENDPOINT}/api/stations?category=country&country=The%20Netherlands"
# Ycast XML calls
curl "http://${ENDPOINT}/setupapp"
# Search by name
curl "http://${ENDPOINT}/ycast/search/?search=Pinguin"
# List top directories (Genres, Countries, Languages, Most Popular).
curl "http://${ENDPOINT}/ycast/radiobrowser/"
# Play station
curl "http://${ENDPOINT}/ycast/play?id=stationid"
# Get station info
curl "http://${ENDPOINT}/ycast/station?id=stationid"
curl "http://${ENDPOINT}/ycast/icon?id=stationid"
# Get station icon

58
examples/filter.yml.example
View file

@ -1,7 +1,53 @@
blacklist:
favicon: ''
# Filters can be applied to the search results coming back from
# api.radio-browser.info. Results can either be whitelisted or blacklisted. The
# attributes in the whitelist are actually the attributes of the Station Struct
# defined here: https://de1.api.radio-browser.info/#Struct_station. The most
# useful ones to filter on are: codec, bitrate, language, languagecode. Ycast
# has a default whitelist for the lastcheckok attribute to be set (1) which
# indicates the stream is currently operational, as it does not make sense to
# return stations to the AV receiver that are broken. There are a few
# attributes that have a multi value string, the values are separated by a
# comma (,) (imo this should be a json list so clients don't have to parse the
# string, but it is as it is). The filter code will split these strings into a
# list first and then will try to match the the value from the whitelist or
# blacklist on any of the values in the list. The most interesting multi value
# attribute is the tags attribute which carries the genre(s) of the station.
# Unfortunately the values are rather free format, so there is no fixed list of
# genres to filter on and most stations indicate multiple genres. Attribute
# filter values can be either a single or multi-value. Multi-values
# should be entered in the filter file as a json list, either using the bracket
# ([]) or list (-) syntax. See the examples below.
#
# For the directory listings by Genre, Language, Country the following filter
# attributes will be applied: Genre: tags; Language: languagecodes; Country:
# country.
whitelist:
country: Germany
countrycode: DE,US,NO,GB
languagecodes: en,no
lastcheckok: 1
#Filter on the full country name:
#country: Germany
#Filter on any of the country codes specified in this [] list:
#countrycode: [ "DE","US","NO","GB" ]
#Filter on any of the language codes specified in this - list:
#languagecodes:
# - "en"
# - "no"
# Filter on bitrate:
#bitrate: 192
#To override the lastcheckok default (1) use this:
#lastcheckok: [ 0,1 ]
blacklist:
# Filter out stations with no favicon:
favicon: ''
# Filter on codecs:
#codec: ["AAC", "AAC+"]
# Limits can be applied, below are the hardcoded defautls, which can be overridden in this file.
#limits:
# The following limits will be applied to the directory listing of genre, country and language. Each item should contain this minium amount of entries to be returned.
#MINIMUM_COUNT_GENRE: 40
#MINIMUM_COUNT_COUNTRY: 5
#MINIMUM_COUNT_LANGUAGE: 5
# The default maximum amount of entries to return from search and search by
# votes.
#DEFAULT_STATION_LIMIT: 200
# Include broken stations in the result.
#SHOW_BROKEN_STATIONS: False

2
ycast/__init__.py
View file

@ -1 +1 @@
__version__ = '1.2.4'
__version__ = '1.3.0'

40
ycast/my_filter.py
View file

@ -20,13 +20,19 @@ def init_filter_file():
filter_dictionary = {}
is_updated = True
if 'whitelist' in filter_dictionary:
# Copy dict items to keep the 1 default item
for f in filter_dictionary['whitelist']:
white_list[f]=filter_dictionary['whitelist'][f]
if filter_dictionary['whitelist'] is None:
white_list = { "lastcheckok" : 1}
else:
# Copy so the default is preserved.
for f in filter_dictionary['whitelist']:
white_list[f]=filter_dictionary['whitelist'][f]
if 'blacklist' in filter_dictionary:
# reference, no defaults
black_list = filter_dictionary['blacklist']
if filter_dictionary['blacklist'] is None:
black_list={}
else:
black_list=filter_dictionary['blacklist']
if 'limits' in filter_dictionary:
set_limits(filter_dictionary['limits'])
@ -68,14 +74,24 @@ def parameter_failed_evt(param_name):
def verify_value(ref_val, val):
if ref_val == val:
return True
if ref_val is None:
return len(val) == 0
if type(val) is int:
return val == int(ref_val)
if val:
return ref_val.find(val) >= 0
if isinstance(val, str) and val.find(",") > -1:
val_list=val.split(",")
else:
val_list=[val]
for v in val_list:
if v == None:
v=''
if isinstance(ref_val, list):
return v in ref_val
if str(ref_val) == str(v):
return True
if ref_val is None:
return len(v) == 0
# if type(val) is int:
## return val == int(ref_val)
# if val:
# return ref_val.find(str(val)) >= 0
return False

5
ycast/radiobrowser.py
View file

@ -88,7 +88,6 @@ def get_station_by_id(vtune_id):
def get_country_directories():
begin_filter()
country_directories = []
apicall = 'countries'
if not get_limit('SHOW_BROKEN_STATIONS'):
@ -104,7 +103,6 @@ def get_country_directories():
def get_language_directories():
begin_filter()
language_directories = []
apicall = 'languages'
if not get_limit('SHOW_BROKEN_STATIONS'):
@ -129,7 +127,8 @@ def get_genre_directories():
for genre_raw in genres_raw:
if get_json_attr(genre_raw, 'name') and get_json_attr(genre_raw, 'stationcount') and \
int(get_json_attr(genre_raw, 'stationcount')) > get_limit('MINIMUM_COUNT_GENRE'):
genre_directories.append(generic.Directory(get_json_attr(genre_raw, 'name'),
if my_filter.chk_parameter('tags', get_json_attr(genre_raw, 'name')):
genre_directories.append(generic.Directory(get_json_attr(genre_raw, 'name'),
get_json_attr(genre_raw, 'stationcount'),
get_json_attr(genre_raw, 'name').capitalize()))
return genre_directories

66
ycast/test_YCast.py
View file

@ -18,19 +18,23 @@ class MyTestCase(unittest.TestCase):
def test_verify_values(self):
assert my_filter.verify_value(None, None)
assert my_filter.verify_value('', None)
assert my_filter.verify_value('', '')
assert my_filter.verify_value(None, '')
assert my_filter.verify_value(3, 3)
assert my_filter.verify_value('3', 3)
assert my_filter.verify_value('3', '3')
assert my_filter.verify_value('3,4,5,6', '5')
assert my_filter.verify_value('3', '3,4,5')
assert my_filter.verify_value(['3','5'], '3')
assert my_filter.verify_value(['3','5'], '3,6')
assert my_filter.verify_value([3,4,5,6], 5)
assert not my_filter.verify_value('', None)
assert not my_filter.verify_value('', '3')
assert not my_filter.verify_value(3, 4)
assert not my_filter.verify_value('3', 4)
assert not my_filter.verify_value('4', '3')
assert not my_filter.verify_value('3,4,5,6', '9')
assert not my_filter.verify_value(['3,4,5,6'], '9')
assert not my_filter.verify_value(['3,4,5,6'], '9,8')
def test_init_filter(self):
my_filter.begin_filter()
@ -44,26 +48,72 @@ class MyTestCase(unittest.TestCase):
else:
logging.warning(" <empty list>")
def test_life_popular_station(self):
def test_station_search(self):
# hard test for filter
stations = radiobrowser.get_stations_by_votes(10000000)
my_filter.white_list={}
my_filter.black_list={}
stations = radiobrowser.search('Pinguin Pop')
logging.info("Stations found (%d)", len(stations))
assert len(stations) == 1
my_filter.white_list={}
my_filter.black_list={ "countrycode": 'NL'}
stations = radiobrowser.search('Pinguin Pop')
logging.info("Stations found (%d)", len(stations))
assert len(stations) == 0
def test_station_by_country(self):
my_filter.white_list={ "codec" : "OGG" }
my_filter.black_list={ }
stations = radiobrowser.get_stations_by_country('Germany')
logging.info("Stations (%d)", len(stations))
# Currently yields 40 but is not fixed of course
assert len(stations) > 20 and len(stations) < 70
def test_station_by_language(self):
my_filter.white_list={ "codec" : "AAC"}
my_filter.black_list={"countrycode": "NL"}
stations = radiobrowser.get_stations_by_language('dutch')
logging.info("Stations (%d)", len(stations))
# With this filter there is only 1 (atm).
assert len(stations) == 1
def test_station_by_genre(self):
my_filter.white_list={"bitrate" : 320}
my_filter.black_list={}
stations = radiobrowser.get_stations_by_genre('rock')
logging.info("Stations (%d)", len(stations))
# Currently yields 86 but is not fixed of course
assert len(stations) > 50 and len(stations) < 100
def test_station_by_votes(self):
my_filter.white_list={}
my_filter.black_list={}
stations = radiobrowser.get_stations_by_votes()
logging.info("Stations (%d)", len(stations))
assert len(stations) == my_filter.get_limit('DEFAULT_STATION_LIMIT')
#stations = radiobrowser.get_stations_by_votes(10000)
#logging.info("Stations (%d)", len(stations))
#assert len(stations) == 10000
def test_get_languages(self):
my_filter.white_list={ 'languagecodes' : ['en','no'] }
my_filter.black_list={}
result = radiobrowser.get_language_directories()
logging.info("Languages (%d)", len(result))
# Based on the example filter is should yield 2
assert len(result) == 2
def test_get_countries(self):
my_filter.init_filter_file()
# Test for Germany only 1, nach der Wiedervereinigung...
my_filter.white_list={ 'country' : 'Germany' }
my_filter.black_list={}
result = radiobrowser.get_country_directories()
logging.info("Countries (%d)", len(result))
# Test for Germany only 1, nach der Wiedervereinigung...
assert len(result) == 1
def test_get_genre(self):
my_filter.white_list={ 'tags' : ['rock','pop'] }
my_filter.black_list={}
result = radiobrowser.get_genre_directories()
logging.info("Genres (%d)", len(result))
# Not a useful test, changes all the time