Autumn 2015

TIES456 Introduction to SOA and Cloud Computing, 5 ECTS

TIES532 - Service oriented architectures and cloud computing for developers, 5 ECTS

Today

• Review of week 42-43
• Small test
• Feedback
• Some extra’s about caching?

Review week 42 - overview

Test – feedback

Discussion about reflective questions

1. How would you implement a page which shows a list of contacts. See also List keys.
2. Why would you ever use a set-up with virtual machines in a real (production) environment? Or would you not?
3. Which of the optimizations made sense, which ones not?
4. What should be improved in this exercise if it is given to students in the future?

Listing keys

• Use a different kind of database

Virtual machines in production

• Benefits
  • easy deployment (you can define templates)
  • improve utilization rate by putting multiple VM on one real server - cost effective
  • security (restrict access)
    • Isolated environment for testing things
    • Fine grained access control
  • Some protection against hardware failure
  • State can be saved in a file and the whole machine can be moved
  • Fast ‘network’ connection between machines
    • You can try bad connections too
• Drawback
  • For the absolute performance virtual machines are not the best, but security wise they make a lot of sense.
    • HPC is never done using virtual machines
• Note
  • Your http server was only accesible from the host (oksa3), one can set-up port forwarding such that one port of oksa3 is forwarded to a specific port on the VM

Optimizations

• Made sense
  • Change framework
  • Caching (in some cases)
• No difference (also not worse)
  • Protocol buffers
  • Load balance database servers

Thesis topics

• Caching using groupcache could solve some headaches
  • Similar idea to cache from Guava
  • The cache is reponsible for fetching missing data
• Paxos is a protocol used to ensure the finding of a consensus about something between a group of computers.
  A thesis work related to this algorithms is possible.
• Docker is a new way of managing container virtualization. You can make thesis in which you compare different options for virtualization, compare and experiment with their properties.
• …

Review week 42 - caching

• Caches are hard to get right

                @post('/add')
                def add():
                    ... get name and number from form
                    ... add to database
  
                @get('search/<name>')
                def search(name):
                      number = mcClient.get(name)
                      if (number is None):
                          number = riakBucket.get(name)
                          mcClient.set(name, number)
                       return "Some html with the number"

Review week 42 - caching

• Problem : stale data in cache in the following case:
  1. /add?name=A&number=0
     • DB: (A,0)
     • MC: -
  2. /search?name=A //returns 0
     • DB: (A,0)
     • MC: (A,0)
  3. /add?name=A&number=1 //update to 1
     • DB: (A,1)
     • MC: (A,0)
  4. /search?name=A //returns 0
     • DB: (A,1)
     • MC: (A,0)

Review week 42 - caching

• Caches are hard to get right - let’s try to solve this - attempt 1

                @post('/add')
                def add():
                    ... get name and number from form
                    ... remove from memcache
                    ... add to database
  
                @get('search/<name>')
                def search(name):
                      number = mcClient.get(name)
                      if (number is None):
                          number = riakBucket.get(name)
                          mcClient.set(name, number)
                       return "Some html with the number"

Review week 42 - caching

• Problem : stale data in cache in the following (multi-threaded) case:

  1. /add?name=A&number=0

     • DB: (A,0)
     • MC: -

  2. Add and search simultaneously

     /add?name=A&number=1	/search?name=A	DB	MC
     Get data from form		(A,0)	(A,0)
     remove from memcache		(A,0)	(-,-)
     	number = mcClient.get(name)	(A,0)	(-,-)
     	if (number is None):	(A,0)	(-,-)
     	number = riakBucket.get(name)	(A,0)	(-,-)
     	mcClient.set(name, number)	(A,0)	(A,0)
     add to database		(A,1)	(A,0)

     • DB: (A,1)
     • MC: (A,0)

  3. /search?name=A //returns 0

     • DB: (A,1)
     • MC: (A,0)

Review week 42 - caching

• Caches are hard to get right - let’s try to solve this - attempt 2

                @post('/add')
                def add():
                    ... get name and number from form
                    ... add to database
                    ... remove from memcache
  
                @get('search/<name>')
                def search(name):
                      number = mcClient.get(name)
                      if (number is None):
                          number = riakBucket.get(name)
                          mcClient.set(name, number)
                       return "Some html with the number"

Review week 42 - caching

• Problem : stale data in cache in the following (multi-threaded) case:

  1. /add?name=A&number=0

     • DB: (A,0)
     • MC: -

  2. Add and search simultaneously

     /add?name=A&number=1	/search?name=A	DB	MC
     	number = mcClient.get(name)	(A,0)	(A,0)
     	if (number is None):	(A,0)	(-,-)
     	number = riakBucket.get(name)	(A,0)	(-,-)
     	#number == 0	(A,0)	(-,-)
     Get data from form		(A,0)	(-,-)
     remove from memcache		(A,0)	(-,-)
     add to database		(A,1)	(-,-)
     	mcClient.set(name, number)	(A,1)	(A,0)

     • DB: (A,1)
     • MC: (A,0)

  3. /search?name=A //returns 0

     • DB: (A,1)
     • MC: (A,0)

Review week 42 - caching

• You could also do

                @post('/add')
                def add():
                    ... get name and number from form
                    ... add to database
                    ... set on memcache = overwrite

Review week 42 - caching - a problem can still occur

• Problem : stale data in cache in the following (multi-threaded) case:

  1. /add?name=A&number=0

     • DB: (A,0)
     • MC: -

  2. Add and search simultaneously

     /add?name=A&number=0	/search?name=A	DB	MC
     	number = mcClient.get(name)	(A,0)	(A,0)
     	if (number is None):	(A,0)	(-,-)
     	number = riakBucket.get(name)	(A,0)	(-,-)
     	#number == 0	(A,0)	(-,-)
     Get data from form		(A,0)	(-,-)
     set on memcache		(A,0)	(A,1)
     add to database		(A,1)	(A,1)
     	mcClient.set(name, number)	(A,1)	(A,0)

     • DB: (A,1)
     • MC: (A,0)

  3. /search?name=A //returns 0

     • DB: (A,1)
     • MC: (A,0)

Review week 42 - caching

• Caches are hard to get right - let’s try to solve this

                @post('/add')
                def add():
                    ... get name and number from form
                    ... add to database
                    ... remove from memcache
  
                @get('search/<name>')
                def search(name):
                      number = mcClient.get(name)
                      if (number is None):
                          number = riakBucket.get(name)
                          mcClient.cas(name, number) #only do a set if not updated
                       return "Some html with the number"

Review week 42 - caching

• Problem : The teacher cannot find any guarantees on whether the CAS counter is updated on delete. If this is not the case, the following could happen: stale data in cache in the following (multi-threaded) case:

  1. /add?name=A&number=1

     • DB: (A,1)
     • MC: (-,-)

  2. Add and search simultaneously

     /add?name=A&number=2	/search?name=A	DB	MC
     	number = mcClient.get(name)	(A,1)	(-,-)
     	if (number is None):	(A,1)	(-,-)
     	number = riakBucket.get(name)	(A,1)	(-,-)
     Get data from form		(A,1)	(-,-)
     add to database		(A,2)	(-,-)
     remove from memcache		(A,2)	(-,-)
     	mcClient.cas(name, number) #case1	(A,2)	(A,1)
     	mcClient.cas(name, number) #case2	(A,2)	(-,-)

     • case 1
       • The counter is not updated on delete and null == null, then the cas will succeed!
       • DB: (A,2)
       • MC: (A,1)
       3. /search?name=A //returns 1
          • DB: (A,2)
          • MC: (A,1)
     • case 2
       • The counter is updated or null != null, then the cas will fail.
       • In this case the problem seems solved.
       • DB: (A,2)
       • MC: (-,-)
       3. /search?name=A //returns 2
          • DB: (A,2)
          • MC: (A,2)

Review week 42 - caching

• Caches are hard to get right
  • One feasible way seems to put a time-out on your memcache entries and time a request can take.
    • It might still be that old data is returned, but it should be rare and only for limited time.
  • Another way is by using a compare-and-swap operation, where you only overwrite if the data is what you expect.
  • If there are multiple memcache servers, they will not inform each other : remove from all
    • If this is a hot piece of data, your database will get be hit a lot of times with the same request

Review week 42 - caching - another problem

• In search

                if mc.get(name):
                   return mc[name] 

• Or

                mc[name] = fetched.encoded_data
                return mc[name] 

• It might be that the data in memcache is removed in between the two calls.