For many years I have noticed some missing expressivity in Dataflow programming. How do you generalize into reusable components. I have been rolling with one solution for a while (flow-queue) which stored values in a queue and released them one at a time. But it had lots of sharp edges and did not really feel like it was true to the dataflow paradigm.
Anyway, I think I finally cracked it. You can clone dataflow subgraphs and run them in parallel variables. They are isolated, graph like, and can link to common shared nodes. For example, in a dashboard you want the time axis shared across many identical but individually configured widgets.
I got this working! Proper reusable dashboard widgets.
This is almost always Step Functions waiting for a state that never finished, not something that “hung.”
Go straight to the execution event history (not the visual graph) and look for the last event that was emitted. That will tell you exactly what state it is stuck in.
In AWS Step Functions, it’s one of these things 90% of the time:
callback/task token task where SendTaskSuccess was never called on one code path
Wait state with a bad timestamp or input that wasn’t expected
ResultPath and OutputPath erased the payload, so the next state never happened.
The service integration was done outside of Step Functions, but it never sent a signal back.
They don’t just stop for no reason; they’re waiting forever for something very specific. The raw events will quickly show it.