structures Package

base Module

class stream_framework.storage.redis.structures.base.RedisCache(key, redis=None)

Bases: object

The base for all redis data structures

delete()

get_key()

get_redis()

Only load the redis connection if we use it

key_format = 'redis:cache:%s'

redis

Only load the redis connection if we use it

set_redis(value)

Sets the redis connection

hash Module

class stream_framework.storage.redis.structures.hash.BaseRedisHashCache(key, redis=None)

Bases: stream_framework.storage.redis.structures.base.RedisCache

key_format = 'redis:base_hash_cache:%s'

class stream_framework.storage.redis.structures.hash.FallbackHashCache(key, redis=None)

Bases: stream_framework.storage.redis.structures.hash.RedisHashCache

Redis structure with fallback to the database

get_many(fields, database_fallback=True)

get_many_from_fallback(missing_keys)

Return a dictionary with the serialized values for the missing keys

key_format = 'redis:db_hash_cache:%s'

class stream_framework.storage.redis.structures.hash.RedisHashCache(key, redis=None)

Bases: stream_framework.storage.redis.structures.hash.BaseRedisHashCache

contains(field)

Uses hexists to see if the given field is present

count()

Returns the number of elements in the sorted set

delete_many(fields)

get(field)

get_key(*args, **kwargs)

get_many(fields)

key_format = 'redis:hash_cache:%s'

keys()

set(key, value)

set_many(key_value_pairs)

class stream_framework.storage.redis.structures.hash.ShardedDatabaseFallbackHashCache(key, redis=None)

Bases: stream_framework.storage.redis.structures.hash.ShardedHashCache, stream_framework.storage.redis.structures.hash.FallbackHashCache

class stream_framework.storage.redis.structures.hash.ShardedHashCache(key, redis=None)

Bases: stream_framework.storage.redis.structures.hash.RedisHashCache

Use multiple keys instead of one so its easier to shard across redis machines

contains(field)

count()

Returns the number of elements in the sorted set

delete()

Delete all the base variations of the key

delete_many(fields)

get_key(field)

Takes something like field=”3,79159750” and returns 7 as the index

get_keys()

Returns all possible keys

get_many(fields)

keys()

list all the keys, very slow, don’t use too often

number_of_keys = 10

list Module

class stream_framework.storage.redis.structures.list.BaseRedisListCache(key, redis=None)

Bases: stream_framework.storage.redis.structures.base.RedisCache

Generic list functionality used for both the sorted set and list implementations

Retrieve the sorted list/sorted set by using python slicing

get_results(start, stop)

key_format = 'redis:base_list_cache:%s'

max_length = 100

class stream_framework.storage.redis.structures.list.FallbackRedisListCache(key, redis=None)

Bases: stream_framework.storage.redis.structures.list.RedisListCache

Redis list cache which after retrieving all items from redis falls back to a main data source (like the database)

cache(fallback_results)

Hook to write the results from the fallback to redis

get_fallback_results(start, stop)

get_redis_results(start, stop)

Returns the results from redis

Parameters:

• start – the beginning
• stop – the end

get_results(start, stop)

Retrieves results from redis and the fallback datasource

key_format = 'redis:db_list_cache:%s'

overwrite(fallback_results)

Clear the cache and write the results from the fallback

class stream_framework.storage.redis.structures.list.RedisListCache(key, redis=None)

Bases: stream_framework.storage.redis.structures.list.BaseRedisListCache

append(value)

append_many(values)

count()

get_results(start, stop)

key_format = 'redis:list_cache:%s'

max_items = 1000

the maximum number of items the list stores

remove(value)

remove_many(values)

trim()

Removes the old items in the list

sorted_set Module

class stream_framework.storage.redis.structures.sorted_set.RedisSortedSetCache(key, redis=None)

Bases: stream_framework.storage.redis.structures.list.BaseRedisListCache, stream_framework.storage.redis.structures.hash.BaseRedisHashCache

add(score, key)

add_many(score_value_pairs)

StrictRedis so it expects score1, name1

contains(value)

Uses zscore to see if the given activity is present in our sorted set

count()

Returns the number of elements in the sorted set

get_results(start=None, stop=None, min_score=None, max_score=None)

Retrieve results from redis using zrevrange O(log(N)+M) with N being the number of elements in the sorted set and M the number of elements returned.

index_of(value)

Returns the index of the given value

remove_by_scores(scores)

remove_many(values)

values

sort_asc = False

trim(max_length=None)

Trim the sorted set to max length zremrangebyscore