Hands-on session: Key-Value Data Store (NoSQL) ################################################################################ # Redis (Key-Value Data Store) ################################################################################ 1. Let's retrieve the Docker image with pre-installed Redis docker pull sickp/alpine-redis (we are using a lightweight container; the official one can be obtained with "docker pull redis") 2. We can now create an isolated network with a Redis server docker network create redis_network docker run --rm --network=redis_network --name=redis-server sickp/alpine-redis (to persiste the Redis storage we could use a directory outside the container, using a persistent storage: -v [localdir]:[container dir /data]) 3. In another shell we can create a Redis client, connected to the same network docker run --rm --net=redis_network -it sickp/alpine-redis redis-cli -h redis-server 4. We are ready to use Redis. We consider as use case an web application that acts as online music player, so we present the APIs considering different needs of this web application. ======================================== Basic operations ----------------- Key-Value: Strings ------- GET userId1 SET userId1 matteo GET userId1 APPEND userId1 nardelli GET userId1 EXISTS userId1 EXISTS userId2 SETNX usedId2 giovanni SETNX usedId1 giovanni DEL userId1 EXISTS userId1 EXISTS userId2 EXPIRE userId2 3 EXISTS userId2 ----------------- Key-Value: Hashes (of strings) ------- # We need to implement a recommendation system for a radio station that suggests the next song according to the history of played songs for the same genre. To this end, we need to trace the number of reproductions (a counter) for each genre played by the user. To store the counter per genre, we resort to a hashmap. To store such a data structure for each userX, we simply save it under the key "userXcounter". HSET user1counter rock 1 HGET user1counter rock HEXISTS user1counter classic HGET user1counter classic HSET user1counter classic 1 HSET user1counter rock 4 HSET user1counter jazz 2 HSET user1counter pop 1 HEXISTS user1counter classic HDEL user1counter HEXISTS user1counter classic HKEYS user1counter HVALS user1counter ----------------- Key-Value: Sets (of strings) ------- # We also need to store which bands/singers play a specific genre. To this end, we can resort to "sets" and save each bands/singers under a key representing the specific musical genre. We assume that a band can play several genres. We might be interested in selecting bands belonging to multiple genres, or in identifying a selection of bands that play the same kind of music. SADD rock "pink floyd" SADD rock "queen" SADD rock "springsteen" SADD rock "baustelle" SADD jazz "paolo conte" SADD pop "paolo conte" SADD pop "baustelle" SCARD rock SCARD Rock SADD pop "mozart" SREM pop "mozart" SDIFF rock pop SUNION rock jazz SINTER rock pop ----------------- Key-Value: Sorted Sets (of strings) ------- # The recommendation system might learn from the user behavior upon the suggested songs. Therefore, we need to identify the number of reproduction of the suggested genre, so that, in the future, we can suggest the top-K genres that have been suggested and listened by the user. We can use the "sorted sets" to store the number of reproduction of songs per genre, so that the data structure can automatically determines the top-K elements. # elements in a sorted set are take in order: # if B and A are two elements with different score, then A > B if A.score > B.score # if B and A have exactly the same score, then A > B if A string is lexicographically greater than B string. A and B canoot be equal since sorted sets only have unique elements. ZCARD urepr ZADD urepr 1 rock ZADD urepr 1 jazz ZADD urepr 1 pop ZCARD urepr ZREM urepr pop ZCARD urepr ZRANK urepr pop ZRANK urepr rock ZINCRBY urepr 3 rock ZINCRBY urepr 1 pop ZRANK urepr pop ZRANK urepr rock ZRANGE urepr 0 1 ZRANGE urepr 0 -1 ZRANGE urepr 0 -2 ZRANGEBYSCORE urepr 3 10 ----------------- Key-Value: Lists (of strings) ------- # The music player needs to store the playlist for the user. The playlist can be populated by the user by adding tracks while navigating the music store, or it can be populated by the recommendation system. While the music is playing, tracks are popped out from the playlist. We can store in-memory the playlist by using a "list" data structure. RPUSH uplay "time" RPUSH uplay "money" LPUSH uplay "glory days" LLEN uplay LRANGE uplay 0 -1 # negative numbers are offset starting from the end of the list LRANGE uplay -2 -1 # if the start index is larger than the size of the list, Redis returns an empty list LPOP uplay LLEN uplay LREM uplay "glory days" 0 # overrides data or out of range: LSET uplay 1 "we will rock you" LRANGE uplay 0 -1