# Testing DragonflyDB with Bull

**URL:** <https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98>\
**Category:** Dragonfly Technical\
**Created:** [May 30, 2023, 7:57am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98 "2023-05-30T07:57:22Z")\
**Posts on this page:** 17\
**Page:** 2

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 1, 2023, 11:23pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/21 "2023-06-01T23:23:49Z")

</div>

👍

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 1, 2023, 11:23pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/22 "2023-06-01T23:23:57Z")

</div>

Always happy to help test the process 🙂

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 1, 2023, 11:45pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/23 "2023-06-01T23:45:01Z")

</div>

Testing DragonflyDB with Bull

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 2, 2023, 12:36am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/24 "2023-06-02T00:36:55Z")

</div>

Seems this was _partially_ my fault in my program based on how I was distributing status updates, but it still didn’t have this issue on Redis. There’s probably something missing for staggering jobs.

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 3, 2023, 1:57am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/25 "2023-06-03T01:57:46Z")

</div>

Currently testing the new XTRIM support

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 3, 2023, 2:19am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/26 "2023-06-03T02:19:16Z")

</div>

@inclusive-numbat It seems that as of [https://github.com/dragonflydb/dragonfly/commit/cde6adb34d1dd016a03cdb875bf14ba2311d9244](https://github.com/dragonflydb/dragonfly/commit/cde6adb34d1dd016a03cdb875bf14ba2311d9244) , _every_ job is being pushed into waiting and nothing is being processed with this error:

```auto
0|Calckey_social | WARN 6 [queue deliver] failed(Error: job stalled more than allowable limit) id=16664 attempts=0/10 age=1632m to=https://xxx.xxx/inbox

```

(where xxx.xxx is each instance)

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 3, 2023, 2:21am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/27 "2023-06-03T02:21:41Z")

</div>

![image.png](https://canada1.discourse-cdn.com/flex004/uploads/dragonfly/original/1X/ef9fa2edb787e87689550b7e301f4a50b6829e95.png)  
 ![image.png](https://canada1.discourse-cdn.com/flex004/uploads/dragonfly/original/1X/d8aebf9c36807a97b76a0a59b7bb5e6562565686.png)

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 3, 2023, 11:49am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/28 "2023-06-03T11:49:39Z")

</div>

I’m noticing some interesting graph patterns. Normally, Redis would have jobs constantly in delayed/processing and have a steady stream of active jobs.

 ![image.png](https://canada1.discourse-cdn.com/flex004/uploads/dragonfly/original/1X/2da682a581b7b724a8d1fc97db44277990b256e0.png)

---

<div class="post-metadata">

**Author:** ![inclusive-numbat](https://avatars.discourse-cdn.com/v4/letter/i/9f8e36/32.png) [@inclusive-numbat](https://dragonfly.discourse.group/u/inclusive-numbat)\
**Post date:** [June 3, 2023, 6:07pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/29 "2023-06-03T18:07:37Z")

</div>

Can you show the output of 'info all’s command, please?

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 3, 2023, 8:34pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/30 "2023-06-03T20:34:48Z")

</div>

```yaml
127.0.0.1:6379> info all
# Server
redis_version:6.2.11
dfly_version:df-dev
redis_mode:standalone
arch_bits:64
multiplexing_api:iouring
tcp_port:6379
uptime_in_seconds:71
uptime_in_days:0

# Clients
connected_clients:166
client_read_buf_capacity:2039808
blocked_clients:30

# Memory
used_memory:19162048
used_memory_human:18.27MiB
used_memory_peak:19162992
comitted_memory:379453440
used_memory_rss:324980736
used_memory_rss_human:309.93MiB
object_used_memory:10577544
table_used_memory:4228560
num_buckets:100800
num_entries:22441
inline_keys:0
strval_bytes:660288
updateval_amount:4944779
listpack_blobs:0
listpack_bytes:0
small_string_bytes:854000
pipeline_cache_bytes:29987625
maxmemory:13420671795
maxmemory_human:12.50GiB
cache_mode:store
maxmemory_policy:noeviction

# Stats
total_connections_received:166
total_commands_processed:483034
instantaneous_ops_per_sec:7035
total_pipelined_commands:141365
total_net_input_bytes:82385877
total_net_output_bytes:13298040
instantaneous_input_kbps:-1
instantaneous_output_kbps:-1
rejected_connections:-1
expired_keys:9011
evicted_keys:0
hard_evictions:0
garbage_checked:0
garbage_collected:0
bump_ups:0
stash_unloaded:0
oom_rejections:0
traverse_ttl_sec:0
delete_ttl_sec:0
keyspace_hits:320103
keyspace_misses:32474
total_reads_processed:77878
total_writes_processed:46177
async_writes_count:58600
parser_err_count:0
defrag_attempt_total:0
defrag_realloc_total:0
defrag_task_invocation_total:0

# TIERED
tiered_entries:0
tiered_bytes:0
tiered_reads:0
tiered_writes:0
tiered_reserved:0
tiered_capacity:0
tiered_aborted_write_total:0
tiered_flush_skip_total:0

# PERSISTENCE
last_save:1685809990
last_save_duration_sec:0
last_save_file:

# Replication
role:master
connected_slaves:0
master_replid:a6b95962cf72f48db53ac12f8fa477eab032336d

```

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 3, 2023, 8:34pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/31 "2023-06-03T20:34:50Z")

</div>

```yaml
# Commandstats
cmd_BRPOPLPUSH:487
cmd_COMMAND:1
cmd_DEL:2316
cmd_EVAL:189
cmd_EVALSHA:98152
cmd_EXEC:666
cmd_EXISTS:279051
cmd_GET:16231
cmd_HGET:8
cmd_HGETALL:1317
cmd_HINCRBY:248
cmd_HMSET:2556
cmd_HSET:725
cmd_INCR:2518
cmd_INFO:22
cmd_LLEN:42
cmd_LPUSH:1922
cmd_LRANGE:8291
cmd_LREM:425
cmd_MULTI:666
cmd_PEXPIRE:15
cmd_PSETEX:82
cmd_PSUBSCRIBE:42
cmd_PUBLISH:9582
cmd_RPOPLPUSH:697
cmd_RPUSH:223
cmd_SADD:1976
cmd_SCARD:671
cmd_SET:42850
cmd_SISMEMBER:671
cmd_SMEMBERS:8292
cmd_SREM:421
cmd_SUBSCRIBE:55
cmd_ZADD:60
cmd_ZCARD:57
cmd_ZRANGE:399
cmd_ZRANGEBYSCORE:369
cmd_ZREM:694
cmd_ZREMRANGEBYRANK:15
cmd_ZREMRANGEBYSCORE:15
cmd_ZSCORE:15

# Errorstats
Dragonfly does not allow execution of a server-side Lua script inside MULTI/EXEC block:637

# Keyspace
db0:keys=22441,expires=1479,avg_ttl=-1

# CPU
used_cpu_sys:176.23134
used_cpu_user:139.794328
used_cpu_sys_children:0.0
used_cpu_user_children:0.0
used_cpu_sys_main_thread:11.422739
used_cpu_user_main_thread:9.95562

# Cluster
cluster_enabled:0

```

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 3, 2023, 8:35pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/32 "2023-06-03T20:35:05Z")

</div>

as of latest commit

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 3, 2023, 8:39pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/33 "2023-06-03T20:39:24Z")

</div>

And this is from my Calckey config file for relevant Redis/Bull settings:

```yaml
# Redis server config
redis:
  host: [redacted]
  port: 6379

# Number of worker processes
clusterLimit: 8

# Job concurrency per worker
deliverJobConcurrency: 128
inboxJobConcurrency: 64

# Job rate limiter
deliverJobPerSec: 1024
inboxJobPerSec: 512

# Job attempts
deliverJobMaxAttempts: 10
inboxJobMaxAttempts: 4

```

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 3, 2023, 8:40pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/34 "2023-06-03T20:40:13Z")

</div>

Which are referenced here: [https://codeberg.org/calckey/calckey/src/branch/develop/packages/backend/src/queue/queues.ts](https://codeberg.org/calckey/calckey/src/branch/develop/packages/backend/src/queue/queues.ts) , [https://codeberg.org/calckey/calckey/src/branch/develop/packages/backend/src/queue/initialize.ts](https://codeberg.org/calckey/calckey/src/branch/develop/packages/backend/src/queue/initialize.ts)

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 3, 2023, 11:02pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/35 "2023-06-03T23:02:21Z")

</div>

Cc @inclusive-numbat (I think I forgot to tag you)

---

<div class="post-metadata">

**Author:** ![inclusive-numbat](https://avatars.discourse-cdn.com/v4/letter/i/9f8e36/32.png) [@inclusive-numbat](https://dragonfly.discourse.group/u/inclusive-numbat)\
**Post date:** [June 4, 2023, 8:43am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/36 "2023-06-04T08:43:07Z")

</div>

Thank you ! Seems the only obvious issue left to handle is execution of lua scripts from multi/exec blocks.

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 4, 2023, 10:41am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/37 "2023-06-04T10:41:17Z")

</div>

Ah, the lack of that is probably why I’ve been noticing these things: [https://github.com/dragonflydb/dragonfly/issues/782#issuecomment-1575426054](https://github.com/dragonflydb/dragonfly/issues/782#issuecomment-1575426054)

[Previous page](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98.md?page=1)