Rust Programming

8134 readers

1 users here now

founded 5 years ago

MODERATORS

[email protected]

How to run background jobs? (lemmy.world)

submitted 1 year ago* (last edited 1 year ago) by [email protected] to c/[email protected]

17 comments fedilink hide all child comments

Is there any library for the queueing mechanism?

What's used by the most - Cron? But a task or rather script executed by Cron won't access to the context of an application. Meaning, a task will have be an independent unit. Whereas I want is a library to use inside a project such that it'll have access to everything.

Anything similar to Sidekiq exist in Rust?

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago) (1 children)

For my project I just run them. I'm using async so it is just spawning a task in the background, but you can do the same with threads. Don't underestimate the number of threads that you can run on a modern computer.

If you want some sort of throttling you can stuff tasks into a queue and just run N background threads pulling them off an processing them.

If you need durability then you start to have more trouble. This is where I would start looking at a library. IDK if there are any libraries that handle logging, retries and similar, but if not you can probably get the basics down pretty easily.

[–] [email protected] 0 points 1 year ago* (last edited 1 year ago) (1 children)

For my project I just run them.

How would you "just run" a task every 30 minutes? Every 5 hours? Once a day?

[–] [email protected] 2 points 1 year ago (2 children)

std::thread::spawn(|| {
    loop {
        std::thread::sleep(std::time::Duration::from_secs(30*60));
        do_job();
    }
});

Works pretty well. Maybe add a bit of code to crash the whole process on panic or some other logging. Wastes a few KiB of memory per loop but probably not a major issue. Doing this with async will waste only the tiniest amount of memory.

[–] [email protected] 1 points 1 year ago (1 children)

@kevincox How do you stop the job? Do you use channels like in Go?

[–] [email protected] 1 points 1 year ago

It depends. Sometimes you can just put an exit call at the end of main to kill the thread. If you want to attempt graceful shutdown then usually I just use a boolean shutdown flag. Then the loop becomes while !shutdown.get() {

[–] [email protected] 0 points 1 year ago* (last edited 1 year ago) (1 children)

what if one of the calls crashes? how would you re-run it?

[–] [email protected] 1 points 1 year ago (1 children)

Best option is probably to add a wrapper around the thread that re-spawns it. But you can also just catch panics in the loop.

[–] [email protected] 0 points 1 year ago* (last edited 1 year ago)

How would you re-run it multiple times then? An internal should be progressively greater. How would you terminate it if it continues to produce an exception?