Question

I am trying to use gevent to improve my programe to interact with redis, but when use gevent + redis-py in multi threading scenario, i found that the performence is so pool. When i comment the monkey.patch_all(), the performence is good, i cannot understand why.

ENV and dependency

• python: 2.7
• redis-py: 3.0.1
• gevent: 1.2.1

Example code:

from gevent import monkey
monkey.patch_all()   # comment this code, the result will be diffrent

import redis
import time
import threading
import logging

logging.basicConfig(format="%(asctime)s: %(message)s", level=logging.INFO, datefmt="%H:%M:%S")

# Redis will use default connection pool
conn = redis.Redis(host="127.0.0.1", port=6379, socket_timeout=5, socket_connect_timeout=5)

conn.set("key", "value", ex=50000)


def get_key():
    t = time.time()
    conn.get("key")
    t1 = time.time() - t
    if t1 > 1:
        logging.info("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!  get key fron redis to slow %s" % t1)
    else:
        logging.info("----------------------------------  get key cost time is %s " % t1)


for i in range(1000):
    x = threading.Thread(target=get_key, args=())
    x.start()


logging.info("Main    : all done")

Test result

without monkey.patch_all()

from gevent import monkey
# monkey.patch_all()

without monkey.patch_all(), the redis delay is very small.

13:45:10: ----------------------------------  get key cost time is 0.000197887420654 
13:45:10: ----------------------------------  get key cost time is 0.000231981277466 
13:45:10: ----------------------------------  get key cost time is 0.000230073928833 
13:45:10: ----------------------------------  get key cost time is 0.000216960906982 
13:45:10: ----------------------------------  get key cost time is 0.000287055969238 
13:45:10: ----------------------------------  get key cost time is 0.000190019607544 
13:45:10: ----------------------------------  get key cost time is 0.000202178955078 
13:45:10: ----------------------------------  get key cost time is 0.000118970870972 
13:45:10: ----------------------------------  get key cost time is 0.000120878219604 
13:45:10: Main    : all done

with monkey.patch_all()

from gevent import monkey
monkey.patch_all()

the redis delay is very big, and if i adjust the 1000 threading to bigger, the delay will be more bigger.

13:46:19: ----------------------------------  get key cost time is 0.217961072922 
13:46:19: ----------------------------------  get key cost time is 0.217952013016 
13:46:19: ----------------------------------  get key cost time is 0.218060970306 
13:46:19: ----------------------------------  get key cost time is 0.216876029968 
13:46:19: ----------------------------------  get key cost time is 0.216850996017 
13:46:19: ----------------------------------  get key cost time is 0.216836929321 
13:46:19: ----------------------------------  get key cost time is 0.216396093369 
13:46:19: ----------------------------------  get key cost time is 0.216381072998