Aquanode DocsScaling
How many instances an endpoint runs, when it scales to zero, and what keeping one warm costs.
Queue depth, not latency
An endpoint scales on how many calls are waiting, never on how slow a call is. We hold nothing warm by default, so scaling up means renting a machine: a call that arrives with no running instance queues while that rental happens, rather than failing.
Max instances bounds how far an endpoint can scale up. When the queue empties, instances scale back down, to zero by default, so an idle endpoint is not holding a GPU.
Idle timeout
Idle timeout (default 300 seconds) is how long an instance sits with nothing to do before it is released. A shorter timeout gets you to zero sooner; a longer one avoids paying the cold-start cost again for calls that arrive close together.
Keeping one instance warm
The Keep one warm toggle sets a minimum of one running instance instead of zero. Turn it on and a call never has to wait on a cold rental, at the cost of paying for that one instance continuously, even with nothing calling it.
It is off by default. Aquanode never holds a machine warm on your behalf unless you ask for it.
Cold starts
With no instance kept warm, the first call after an idle period has to rent a machine before it can be answered. That is a real wait, not a network round trip, so plan for it rather than for an instant reply.
How long a cold start takes
Expect about 1 to 4 minutes from the call to the first reply. Measured with a small HTTP image: about 1 minute on a container provider and about 3.5 minutes on a full VM. A large image takes longer to pull. Turn on Keep one warm if a cold start is not acceptable for your use case.
Next
- Calling an endpoint: tokens, the URL, and reading a result back