# Testing DragonflyDB with Bull

**URL:** <https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98>\
**Category:** Dragonfly Technical\
**Created:** [May 30, 2023, 7:57am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98 "2023-05-30T07:57:22Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [May 30, 2023, 7:57am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/1 "2023-05-30T07:57:22Z")

</div>

Hey everyone! I’m trying to migrate from Redis to DragonflyDB. I’m using the `ioredis` and `@bull-board/api` packages for Redis.

I set up DragonflyDB on port 6380, and ran `redis-cli -p 6380` to get into it. I then ran `replicaof local 6379`, and `dbsize` showed the correct data. However, I then restarted my app and got met with the error “Failed to handle unlocked job in active”.

What’s the proper way to migrate from Redis to DragonflyDB? I already tried `riot-redis` to no avail.

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [May 30, 2023, 8:02am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/2 "2023-05-30T08:02:13Z")

</div>

@inclusive-numbat apologies for the ping, but I see you’re online ^^;

---

<div class="post-metadata">

**Author:** ![inclusive-numbat](https://avatars.discourse-cdn.com/v4/letter/i/9f8e36/32.png) [@inclusive-numbat](https://dragonfly.discourse.group/u/inclusive-numbat)\
**Post date:** [May 30, 2023, 9:20am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/3 "2023-05-30T09:20:45Z")

</div>

If you are using bullmq underneath, we do not yet support it. See the open issue in the repo

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [May 30, 2023, 10:07pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/4 "2023-05-30T22:07:59Z")

</div>

Ah. Could you point me to the issue?

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [May 30, 2023, 10:10pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/5 "2023-05-30T22:10:07Z")

</div>

Ah, nvm. This one? [https://github.com/dragonflydb/dragonfly/issues/782](https://github.com/dragonflydb/dragonfly/issues/782)

---

<div class="post-metadata">

**Author:** ![inclusive-numbat](https://avatars.discourse-cdn.com/v4/letter/i/9f8e36/32.png) [@inclusive-numbat](https://dragonfly.discourse.group/u/inclusive-numbat)\
**Post date:** [May 30, 2023, 10:27pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/6 "2023-05-30T22:27:02Z")

</div>

Yep

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 1, 2023, 2:10am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/7 "2023-06-01T02:10:09Z")

</div>

@inclusive-numbat so, I built from source and used the `--default_lua_config=allow-undeclared-keys` parameter to get it running. However, it seems that it’s not being utilized efficiently? Is there any tweaking/tuning I can do? I have Dragonfly running on a dedicated 8 core 16GB VPS, but it’s using very little resources and puts many jobs into “waiting” rather than processing them…

 ![image.png](https://canada1.discourse-cdn.com/flex004/uploads/dragonfly/original/1X/42926af7a959058b18a7f7b9738a82f63c8c9cd1.png)

---

<div class="post-metadata">

**Author:** ![inclusive-numbat](https://avatars.discourse-cdn.com/v4/letter/i/9f8e36/32.png) [@inclusive-numbat](https://dragonfly.discourse.group/u/inclusive-numbat)\
**Post date:** [June 1, 2023, 10:25am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/8 "2023-06-01T10:25:12Z")

</div>

@meaningful-vole

---

<div class="post-metadata">

**Author:** ![inclusive-numbat](https://avatars.discourse-cdn.com/v4/letter/i/9f8e36/32.png) [@inclusive-numbat](https://dragonfly.discourse.group/u/inclusive-numbat)\
**Post date:** [June 1, 2023, 10:26am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/9 "2023-06-01T10:26:41Z")

</div>

Try using `--default_lua_config=allow-undeclared-keys,disable-atomicty`

---

<div class="post-metadata">

**Author:** ![inclusive-numbat](https://avatars.discourse-cdn.com/v4/letter/i/9f8e36/32.png) [@inclusive-numbat](https://dragonfly.discourse.group/u/inclusive-numbat)\
**Post date:** [June 1, 2023, 10:27am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/10 "2023-06-01T10:27:00Z")

</div>

but we do not officially support bull, yet

---

<div class="post-metadata">

**Author:** ![inclusive-numbat](https://avatars.discourse-cdn.com/v4/letter/i/9f8e36/32.png) [@inclusive-numbat](https://dragonfly.discourse.group/u/inclusive-numbat)\
**Post date:** [June 1, 2023, 10:27am UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/11 "2023-06-01T10:27:28Z")

</div>

Tell us if it improves things on your end

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 1, 2023, 10:04pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/12 "2023-06-01T22:04:23Z")

</div>

@inclusive-numbat doesn’t seem to work

```auto
I20230601 18:04:00.422749 3223497 init.cc:69] ./dragonfly running in opt mode.
I20230601 18:04:00.423955 3223497 dfly_main.cc:592] Starting dragonfly df-dev-0000000
W20230601 18:04:00.424810 3223497 dfly_main.cc:629] SWAP is enabled. Consider disabling it when running Dragonfly.
I20230601 18:04:00.425529 3223497 dfly_main.cc:632] maxmemory has not been specified. Deciding myself....
I20230601 18:04:00.425748 3223497 dfly_main.cc:636] Found 15.62GiB available memory. Setting maxmemory to 12.50GiB
I20230601 18:04:00.462266 3223504 uring_proactor.cc:158] IORing with 1024 entries, allocated 102720 bytes, cq_entries is 2048
I20230601 18:04:00.494704 3223497 proactor_pool.cc:77] Running 16 io threads
F20230601 18:04:00.494937 3223497 script_mgr.cc:49] Check failed: !err Invalid flag: disable-atomicty
***Check failure stack trace:***
    @ 0x5590d3620000 google::LogMessage::Fail()
    @ 0x5590d36274d7 google::LogMessage::SendToLog()
    @ 0x5590d361fa13 google::LogMessage::Flush()
    @ 0x5590d36213af google::LogMessageFatal::~LogMessageFatal()
    @ 0x5590d32f7beb dfly::ScriptMgr::ScriptMgr()
    @ 0x5590d3313544 dfly::ServerFamily::ServerFamily()
    @ 0x5590d32b41ac dfly::Service::Service()
    @ 0x5590d326164b dfly::(anonymous namespace)::RunEngine()
    @ 0x5590d3258508 main
    @ 0x7f05b6741d90 (unknown)
    @ 0x7f05b6741e40 __libc_start_main
    @ 0x5590d325ce65 _start
***SIGABRT received at time=1685642640 on cpu 11***
PC: @ 0x7f05b67aea7c (unknown) pthread_kill
[1] 3223497 IOT instruction (core dumped) ./dragonfly --default_lua_flags=allow-undeclared-keys,disable-atomicty 

```

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 1, 2023, 10:05pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/13 "2023-06-01T22:05:36Z")

</div>

When running `--help` I get this:

```auto
 Flags from /server/script_mgr.cc:
    --default_lua_flags (Configure default flags for running Lua scripts:
      - Use 'allow-undeclared-keys' to allow accessing undeclared keys,
      - Use 'disable-atomicity' to allow running scripts non-atomically.
      Specify multiple values separated by space, for example
      'allow-undeclared-keys disable-atomicity' runs scripts non-atomically and
      allows accessing undeclared keys); default: "";
    --lua_auto_async (If enabled, call/pcall with discarded values are
      automatically replaced with acall/apcall.); default: false;

```

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 1, 2023, 10:06pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/14 "2023-06-01T22:06:32Z")

</div>

Ah, seems that `./dragonfly --default_lua_flags='allow-undeclared-keys disable-atomicity' --alsologtostderr` works

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 1, 2023, 10:09pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/15 "2023-06-01T22:09:12Z")

</div>

Doesn’t really seem to change anything though…

 ![image.png](https://canada1.discourse-cdn.com/flex004/uploads/dragonfly/original/1X/cdf08b9e16b460dc5cc329acdfd8281f716a53d2.png)

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 1, 2023, 10:09pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/16 "2023-06-01T22:09:46Z")

</div>

Actually if anything it seems to have made things slightly worse

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 1, 2023, 10:19pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/17 "2023-06-01T22:19:57Z")

</div>

[https://github.com/dragonflydb/dragonfly/issues/1335](https://github.com/dragonflydb/dragonfly/issues/1335)

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 1, 2023, 10:21pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/18 "2023-06-01T22:21:40Z")

</div>

@inclusive-numbat I can give you moderator permissions on my site to view bull-board if you want, if that’ll help with solving issues 🙂

---

<div class="post-metadata">

**Author:** ![winsome-nautilus](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@winsome-nautilus](https://dragonfly.discourse.group/u/winsome-nautilus)\
**Post date:** [June 1, 2023, 10:44pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/19 "2023-06-01T22:44:33Z")

</div>

@inclusive-numbat I just noticed something really weird – the Waiting section of my Delivery queue (there’s an Inbox and Delivery queue) just jumped from 0 to 7,000 in a split second, and this never happened on Redis. I wonder if there’s some scheduling issues 😟  
 ![image.png](https://canada1.discourse-cdn.com/flex004/uploads/dragonfly/original/1X/7f3d9ef6a8b263ef019c16e4666138b511927d2c.png)

---

<div class="post-metadata">

**Author:** ![meaningful-vole](https://avatars.discourse-cdn.com/v4/letter/m/bb73d2/32.png) [@meaningful-vole](https://dragonfly.discourse.group/u/meaningful-vole)\
**Post date:** [June 1, 2023, 11:17pm UTC](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98/20 "2023-06-01T23:17:51Z")

</div>

@winsome-nautilus

1. Yes, allow-undeclared-keys makes Dragonfly slower, a lot. Scripts that access undeclared keys are unpredictable. Unpredictability and multi-threading don’t mix well 🙄
2. Roman forgot an `i`: its `disable-atomicity`
3. Please note that from `1.4.0` the flag will change to `default_lua_flags`

We have in our plans to check compatibility with Bull and we’ll check your issues. `disable-atomicity` actually disables atomicity and it means that multiple scripts can operate on the same set of keys at the same time. **If the script requires atomicity for correct execution, it won’t work properly in this mode**. If you’re not sure, please don’t run them with this mode - it can lead to all kinds of undefined behavior and possible bugs.

We’re happy to see you using Dragonfly. However currently we still have some issues with some of the popular frameworks. We hope it’ll improve in the future 🙂

[Next page](https://dragonfly.discourse.group/t/testing-dragonflydb-with-bull/98.md?page=2)
