Cut my AI API bill from $310 to $47 by caching answers people asked before
I've been running a little chatbot for my brother's fishing forum since March, and the API bill hit $310 one month because everyone kept asking the same 20 starter questions. I set up a simple cache on the server that saves each answer to a file the first time it's generated, then just serves that file next time instead of calling the model again. Over 6 weeks my bill dropped to $47 total, and about 68% of all questions were repeats of stuff already answered. The one catch is that cached replies can go stale, so I wipe the folder every Sunday night and let it rebuild fresh. I know this is basic stuff to most folks here, but it worked way better than I expected. Anyone got a smarter way to handle repeat questions without making the bot feel copy pasted?