Empty results with redis caching enable

[systemofapwne]

When having redis caching enabled, the response of steamapi is an empty json cluster

{"data": {"730": {}}, "status": "success"}

The api prints theses logs

level=INFO msg="Requested app info" apps=[730]
level=INFO msg="Redis connection pool created successfully"
level=INFO msg="App info could not be found in cache" apps=[730]
level=INFO msg="No app has been found at Steam but the request was succesfull" apps=[730]
10.6.0.4:54858 - "GET /v1/info/730 HTTP/1.1" 200

This bug looks like a regression, introduced by #eab6856 : With lines 45-47 removed, there will never be a call to the steam, no data will be stored in redis so the response will always be an empty JSON.

Turning off caching makes the api behave as expected.

I propose to add the missing lines back.

[jonakoudijs]

Thank you for creating the issue. This change was temporary and ment to fix major instability in the API. But as many temporary things.. they become permanent. The outage was caused by some people trying to retrieve all data systematically but by doing so, overloading the platform.

In short, they simply tried doing a GET on every number between 0 and infinite (?). At some point the highest app id number was reached. The API kept trying to retrieve app info for these non-existing app's and storing that empty result in the cache. Because both the API was constantly retrieving data instead of using the cache and the cache contained a lot of results, this caused immediate issues and prompted me to cut some code.

In its current state the steam Python library cannot see the difference between a non-existing Steam app or an incorrect result from Steam (or something else going during processing). This makes catching incorrect requests quite hard.

Hopefully this gives some clarity on why this code was cut as an emergency change.

---

Coming back to your issue. I can see the benefit of having the API directly requesting app info when not using the background worker or in the case of a recently cleared cache. Because it can cause issues in the current SteamCMD API environment I would suggest that we add an option to the code that enables/disables this functionality.

If you are able to wip up something for this, that would be great. Personally I am largely unavailable the coming period, so I can't give guarantees on when I could work myself on it.

Also perhaps good to know; in the API rewrite I am currently working on (when I do have time..), data is only retrieved with a background worker and only the "frontend" api is only used to retrieve app info from the cache. To check if a requested app id is valid it is checked against a different list of id's that are periodically retrieved via a different Steam endpoint.

[systemofapwne]

Coming back to your issue. I can see the benefit of having the API directly requesting app info when not using the background worker or in the case of a recently cleared cache.

Tell me more about this worker. I see, that you created a job service that fills and updates the cache via tasks. I was trying to scavange for documentation but it was kind of sparse. The link to the platform repo is unfortunately dead.

Because it can cause issues in the current SteamCMD API environment I would suggest that we add an option to the code that enables/disables this functionality.

If you are able to wip up something for this, that would be great. Personally I am largely unavailable the coming period, so I can't give guarantees on when I could work myself on it.

If I find the time and motivation, that should be kind of easy. But I guess, having the documentation for how to properly configure the workers makes more sense here.

Also perhaps good to know; in the API rewrite I am currently working on (when I do have time..), data is only retrieved with a background worker and only the "frontend" api is only used to retrieve app info from the cache. To check if a requested app id is valid it is checked against a different list of id's that are periodically retrieved via a different Steam endpoint.

Sounds nice. I can also just wait for the rewrite. Right now I have 1.15.1 deployed is doing, what I need (with caching enabled)

[jonakoudijs]

Apologies for the late reply. Yes, I have not made much documentation because it used to be quite buggy. And while it is still not perfect it is relatively stable in the past few months. My current plan is to document everything in a more proper way after the rewrite to avoid extra work, seeing the rewrite changes quite a lot.

The worker can be started by running the same steamcmd / api image and overwriting the entrypoint and command like this:

 docker run --entrypoint="celery" -ti steamcmd/api:latest -A job worker --loglevel=info --concurrency=10 --beat --schedule /tmp/celerybeat-schedule --pidfile /tmp/celery.pi

Alternatively if you use Kubernetes your Deployment manifest could look something like this:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: steamcmd-job
  labels:
    app: steamcmd-job
spec:
  replicas: 1
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
  selector:
    matchLabels:
      app: steamcmd-job
  template:
    metadata:
      labels:
        app: steamcmd-job
    spec:
      containers:
        - name: steamcmd-job
          image: steamcmd/api:latest
          env:
            - name: CACHE
              value: "True"
            - name: CACHE_TYPE
              value: redis
            - name: CACHE_EXPIRATION
              value: "0"
            - name: REDIS_URL
              value: "redis://:xxxxxxxxxxxxxxxxx@your-redis-service-name:6379/0"
            - name: PYTHONUNBUFFERED
              value: "1"
            - name: LOG_LEVEL
              value: info
          command: ["celery"]
          args: ["-A", "job", "worker", "--loglevel=info", "--concurrency=10", "--beat", "--schedule", "/tmp/celerybeat-schedule", "--pidfile", "/tmp/celery.pid"]
          resources:
            requests:
              memory: "512Mi"
              cpu: "250m"
            limits:
              memory: "1024Mi"

As you can see in this manifest, you will need to define a few variables just like the (web) API. Here Redis is also used as a message broker and not just as a cache for the app information. Also you can run multiple instances of the Celery worker at the same time, they will simply divide the jobs between themselves.

In its current form the worker performs a check for updated app's in the past few seconds and triggers the retrieval of those app's. The same Celery instance(s) will then retrieve the app info and store them in the Redis cache like the web API does. Let me know if you need any help or have any other questions.