You are viewing a single comment's thread from:

RE: An evening with Python, Multi-threading, STEEM and Beem

in #programming6 years ago

This also seems like a good place where something simpler than a spin lock could be useful: a simple queue.

After all, the posting of comments is a single conceptual entity that must remain sequential because of the wait. There is no need to parallelize it because it can't function in parallel and it doesn't involve any processing which would take longer than three real-world seconds.

So instead of having things spin on a mutex, it's conceptually simpler to have the streaming monitor (which does do enough computation to require its own thread) be able to throw comment IDs up on a "blackboard" list from which a relatively straightforward comment poster thread function takes the first, up votes, makes its log entry, waits three seconds, and then does it all over again forever. If the blackboard is empty, do nothing and go straight to the wait.

This as the main advantage of not requiring a thread pool at all with a predefined number of workers. The maximum amount of concurrency this flow can possibly have is two threads in a reasonable sense so there's no real sense in having more than that.

(I'm pretty sure that concurrent has some built-in magic for shared variables probably using a mutex under the hood, but there's no need to reimplement the wheel, really. Then again, I may have been spoiled by years of writing in Erlang for concurrency, where passing information between independent threads is built into the base of the language via a straightforward message passing system.)

Sort:  

I'm pretty sure there are libraries to implement a similar messaging system in Python.

It's basically the same thing as a web spider, and I'm pretty sure I heard it mentioned when reading a tutorial on writing web spiders. They might have even mentioned Erlang...but I can't remember for sure. Maybe I should try implementing one of these...but the only use cases I have in mind are a bit large of an undertaking.

Every large problem is an accumulation of interesting small problems, especially if you are looking for something to write for an audience.

Why not think about some sort of more ambitious undertaking and show how it's done?

Coin Marketplace

STEEM 0.28
TRX 0.12
JST 0.032
BTC 62521.03
ETH 3025.04
USDT 1.00
SBD 3.97