In one of our current Project we used Memcached. Objective was to minimized database to-fro for very high UDP requests hitting Java application. Hence all device ids were placed in Memcached so that we could very promptly identify the incoming UDP request is coming from valid device or not. 

Memcached is a general-purpose distributed memory caching system. It is often used to speed up dynamic database-driven websites by caching data and objects in RAM to reduce the number of times an external data source must be read. Memcached is free and open-source software.

Architecture of application had IOT Devices at client end which were sending very high packets to Java Application which was running on UDP Port. Java Application was further sending these reading packets and alarm data etc. to Amazon Kinesis stream which was using Lambda function to create records for same in PostgreSQL database.

PROBLEM:

But are nightmare started when memcached cache was getting deleted quite often in week or so and we didn't knew why that was happening. System was not working as expected as IOT devices were receiving error packets as the request device mac id was not available in memcached(which actually got cleaned and didn't had any key in it). Though we were storing the key value indefinitely but its redemption was not know to us. Are analysis of logs and stats of memcached didn't result any success. Though we did temporarily added fixed to recreate memcached in case we found it got deleted, but it was not clean solution. 

Our further brainstorming, figuring out the clean solution led to below: 

SOLUTION:

There was voice that instead of using Memcached we could use its rival Redis. As we were doing further research on it we found Redis cache does get cleared when we do server clear cache as it get linked to it. When we did same test for Memcached (server clear cache) we found that was the real reason for memcached being getting cleaned. 

Actually whenever team was doing clear cache on laravel server using following commands:

ubuntu@ip-172-31-29-151:/var/www/html/signatrol$ php artisan config:clear
Configuration cache cleared!
ubuntu@ip-172-31-29-151:/var/www/html/signatrol$ php artisan config:cache
Configuration cache cleared!
Configuration cached successfully!
ubuntu@ip-172-31-29-151:/var/www/html/signatrol$ php artisan cache:clear
Cache cleared successfully.

It was also clearing the memcached as it was linked to the project.

Latest available version of Memcached 1.5.12 had feature to prevent cache flushing using -F when started with that option, we did clear cache with above and it didn't cleared the memcached. But when we started Memcached server without -F and runned above clear cache command, it was clearing the memcached.

Initially Memcached we were using on Ubuntu 14.04.5 server was not the latest version, by default Memcached which got installed was of version 1.4.14 though latest version 1.5.12 existed. 
First time Memcached was installed using following command, which installed the version 1.4.14
        

apt-get install memcached

 
Hence when we deployed the latest version of Memcached using below steps we didn't had a issue thereafter even after running clear cache on server.

sudo apt-get install libevent-dev
wget http://memcached.org/latest
sudo mv latest memcached-1.5.10.tar.gz
sudo tar -zxvf memcached-1.5.10.tar.gz
cd memcached-1.5.10
./configure && make && make test && sudo make install
cd /usr/local/src/memcached-1.5.10
memcached -d -M -v -F -m 1024 -p 11211