Need best practices guides for performance and memory usage

[ClearScriptLib]

[danbopes]

I can't stress this enough. I'm trying to integrate this into my project, and I'd like to safely add usage and resource limits. What areas I'm struggling with:

• How the V8RuntimeConstraints worked, and some sample documentation on how to properly set reasonable size limits for user run code.
• How do I limit execution time? With jint, I can specify a cancellation token to cancel the currently running script.
• How to kill processes that hog the CPU:

engine.Execute(@"
    while (true) {}
");

• How the engine works with asynchronous code:

public class TestApi
{
    private delegate void Executor(dynamic resolve, dynamic reject);
    private static async Task<string> Delay(string msg)
    {
        await Task.Delay(2000);

        return msg;
    }
    public static object delay(string msg)
    {
        var task = Delay(msg);
        var ctor = (ScriptObject)ScriptEngine.Current.Script.Promise;
        return ctor.Invoke(true, new Executor((resolve, reject) => {
            task.ContinueWith(t => {
                if (t.IsCompleted)
                {
                    resolve(t.Result);
                }
                else
                {
                    reject(t.Exception);
                }
            });
        }));
    }
}

engine.AddHostType("Console", typeof(Console));
engine.AddHostType("api", typeof(TestApi));
engine.Execute(@"
    Console.WriteLine('start');

    const main = async () => {
      Console.WriteLine('main');
      var res = await api.delay('end');
      Console.WriteLine(res);
    }
    main();
");

If I wrap this in a using block, it will output:

start
main

and then dispose of the engine

If I execute this, and add a Console.ReadKey(); before the engine gets disposed, after 2 seconds it correctly outputs end. There's no documentation on how to wait until execution of all callbacks is completed.

[ClearScriptLib]

Hi @danbopes,

• How the V8RuntimeConstraints worked, and some sample documentation on how to properly set reasonable size limits for user run code.

We generally recommend that applications stay away from V8RuntimeConstraints.

Originally this API was somewhat suitable for sandboxing, but the V8 team decided years ago that constrained script execution isn't a goal for V8. Today, exceeding any of the specified limits causes V8 to crash instantly, and that's by design.

The API might still be useful for expanding V8's default limits, but we've found that V8 can then hit other limits that are inaccessible and vary unpredictably from version to version.

ClearScript does offer support for "soft limits" (see MaxRuntimeHeapSize, MaxRuntimeStackUsage), but as they require external monitoring via periodic timers, we can't guarantee their effectiveness in all cases.

The bottom line is that, unfortunately, if you must run untrusted script code with 100% safety, you'll need a Chromium-like multi-process architecture. The V8 team equates executing untrusted script code with invoking a function in an untrusted DLL.

V8's API is constantly changing though, and we'll continue to watch for new developments.

• How do I limit execution time? With jint, I can specify a cancellation token to cancel the currently running script.
• How to kill processes that hog the CPU

For these purposes you can use ScriptEngine.Interrupt.

There's no documentation on how to wait until execution of all callbacks is completed.

There's no special API for that, but you can always wait for a pre-arranged completion signal:

engine.AddHostType("Console", typeof(Console));
engine.AddHostType("api", typeof(TestApi));
engine.Script.done = new ManualResetEventSlim();
engine.Execute(@"
    Console.WriteLine('start');
    const main = async () => {
      Console.WriteLine('main');
      var res = await api.delay('end');
      Console.WriteLine(res);
      done.Set();
    }
    main();
");
engine.Script.done.Wait();

Please send any additional questions or comments our way.

Thanks!

[simontom]

Hey everyone, I'm wondering how to "correctly" set the memory limits and this seemed like the best place to ask based on what I've found. But copying it here as well....

What properties I've found, so far:

• V8ScriptEngine.MaxRuntimeHeapSize

• V8ScriptEngine.MaxRuntimeStackUsage

• V8RuntimeConstraints.MaxArrayBufferAllocation

• V8RuntimeConstraints.MaxOldSpaceSize

• V8RuntimeConstraints.MaxArrayBufferAllocation

• V8ScriptEngine.RuntimeHeapSizeSampleInterval

What properties I use + the values:

• V8ScriptEngine.MaxRuntimeHeapSize = 1GB
• V8RuntimeConstraints.MaxArrayBufferAllocation = 1GB

Properties with default:

• V8RuntimeConstraints.MaxOldSpaceSize = 1.5GB

Question:
Do I need to set explicitly the sample interval?

• V8ScriptEngine.RuntimeHeapSizeSampleInterval
  Or it works per se (has some value implicitly) when the
• V8ScriptEngine.MaxRuntimeHeapSize = 1GB
  is set?

The descriptions seem a bit vague to me, and I miss the default values in the documentation for them 😿

My current playground / flow:

• container (with dotnet) in AKS has a 2GB RAM limit (is killed when engine(s) exceed this limit) which runs the engine(s)
• engine is spawned for each custom JavaScript request and is disposed when the script finishes
• it can run up to 2 JavaScripts (engines) at the same time
• I currently use HeapConsumptionProbe that compares engine.GetRuntimeHeapInfo().TotalHeapSize > engine.MaxRuntimeHeapSize.ToUInt64() => throw which is run periodically (every 10s)

What problem I'm trying to kinda diminish:

• have correct/better settings for memory limits of ClearScript (V8Engine) => lower the number of containers's OOM kills
• stop the engine before it consumes all the RAM

Could give me a hand, please, and help me understand a bit more (find more proper values)?

With kind regards,
Šimon

P.s.: From what I've currently read here, we had wrong expectations / assumptions

[ClearScriptLib]

Hi Šimon,

Do I need to set explicitly the sample interval?

The default value (zero) selects an internally enforced minimum, which is currently 50ms. Our recommendation is not to increase unless you encounter performance issues.

Could give me a hand, please, and help me understand a bit more (find more proper values)?

This probably won't be very helpful, but our understanding is as follows:

• The V8 heap consists of two regions – the young generation, which holds recently allocated objects, and the old generation, which contains objects that have survived several rounds of garbage collection.
• The young generation is further subdivided equally into two semi-spaces and a large object space.
• The heap region and space in which an object resides determines various aspects of its handling by the garbage collector, which can move objects from one region to another.
• The young generation is typically much smaller than the old generation. On 64-bit systems, V8's per-runtime default limits are around 8MB for the former and 1.4GB for the latter, although optional features, when enabled, can alter them.

Ultimately, your ideal V8ResourceConstraints and MaxRuntimeHeapSize numbers depend on the scripts you need to run. We understand that the memory requirements of user-provided scripts are difficult to predict, especially if those scripts could be buggy or even malicious, but that's the unfortunate reality.

A potentially useful feature not mentioned above is on-demand heap expansion, which provides a way to increase the heap size limit when a script comes close to exceeding it. See HeapExpansionMultiplier.

Good luck!

[simontom]

Hello, thanks a lot!
Well, 50ms period seems fine so far. May be, I can/will try to tweak it (lower the period) a little to let it hit sooner.

What is the connection between V8ResourceConstraints.MaxArrayBufferAllocation and MaxRuntimeHeapSize?
Per the DOC, these are set independently (that's why I set both values).
How does is co-operate?
Does the V8ScriptEngine.RuntimeHeapSizeSampleInterval check both the heap and the ArrayBufferAllocation?
If not, how to see/check/validate the allocated memory for ArrayBufferAllocation?

What can be the max memory footprint of the engine if I have these values set:
V8ScriptEngine.MaxRuntimeHeapSize = 1GB
V8RuntimeConstraints.MaxArrayBufferAllocation = 1GB
V8RuntimeConstraints.MaxOldSpaceSize = 1.5GB

Is it recommended to explicitly call the GC on the V8 Engine before the instance gets dispose?

About the HeapExpansionMultiplier
Is the default value 1? As I understand it, 1 means it does not expand/change the size of the heap?
I think it is not usable for this solution, or ...?

And one more
Is there any difference creating the engine this way

then disposing the engine

vs

var runtime = new V8Runtime("xxx");
var engine = runtime.CreateScriptEngine(scriptEngineFlags);

then

engine.Dispose();
runtime.CollectGarbage(true);
engine = Runtime.CreateScriptEngine( .... );

?

[ClearScriptLib]

Hello Šimon,

What is the connection between V8ResourceConstraints.MaxArrayBufferAllocation and MaxRuntimeHeapSize?

They're unrelated. Memory for ArrayBuffer storage is allocated on the native heap rather than the V8 JavaScript heap.

Does the V8ScriptEngine.RuntimeHeapSizeSampleInterval check both the heap and the ArrayBufferAllocation?

No. Unlike failure to allocate memory on the JavaScript heap, V8 handles ArrayBuffer memory allocation failure cleanly – that is, it simply throws a JavaScript exception. ClearScript can therefore enforce that limit without periodic sampling.

If not, how to see/check/validate the allocated memory for ArrayBufferAllocation?

Although ClearScript tracks that internally, there's no API for retrieving it. Sorry!

What can be the max memory footprint of the engine

That's a difficult question to answer. Ignoring code, each V8 runtime includes managed data, native data (including ArrayBuffer storage), and a JavaScript heap.

Our understanding is that V8 crashes the process if any region of the JavaScript heap exceeds the corresponding V8RuntimeConstraints limit, unless the host allows the heap to expand at the last moment (which in ClearScript is done via HeapExpansionMultiplier). It is unclear how heap expansion updates the individual region limits.

V8 itself is unaware of MaxRuntimeHeapSize. When it's set, ClearScript periodically compares it against V8's total heap size (presumably the sum of the old and young/new region sizes) and cleanly terminates the script if necessary. Because periodic monitoring can't guarantee protection against memory usage spikes, we recommend augmenting it with HeapExpansionMultiplier.

There are other factors as well. For example, each V8 runtime reserves a significant chunk of address space up front, which can be a problem on smaller (32-bit) systems that use multiple runtimes in a single process. Last time we looked into it, V8 also had a number of internal limits that the host couldn't control or monitor, but we believe that recent versions have improved on that.

Is the default value 1? As I understand it, 1 means it does not expand/change the size of the heap?

The default value of HeapExpansionMultipler is zero. Set it to a value greater than 1 to enable on-demand heap expansion. We recommend a value of 1.25 or higher.

Is there any difference creating the engine this way

If you're only using one engine, there's no difference. The V8ScriptEngine constructor creates an engine with a private runtime, whereas the V8Runtime constructor and CreateScriptEngine method allow you to use multiple engines that share a runtime.

Please don't hesitate to send additional questions and comments our way!

Cheers!

[simontom]

They're unrelated. Memory for ArrayBuffer storage is allocated on the native heap rather than the V8 JavaScript heap.

No. Unlike failure to allocate memory on the JavaScript heap, V8 handles ArrayBuffer memory allocation failure cleanly – that is, it simply throws a JavaScript exception. ClearScript can therefore enforce that limit without periodic sampling.

Although ClearScript tracks that internally, there's no API for retrieving it. Sorry!

Well, this sounds really good. So, when the ArrayBuffer allocation rises up to the limit, the script fails. Thanks to u, I don't need any xtra polling or whatever to check/handle that.

However, the max memory limit it can really reach (by my understanding) is "max heap limit + old heap limit + ArrayBuffer allocation limit".
Is there any plan to somehow merge it to common limit to allow easier setup?

V8 itself is unaware of MaxRuntimeHeapSize. When it's set, ClearScript periodically compares it against V8's total heap size ....

Thanks for the cleaner explanation of how CS handles this.

The default value of HeapExpansionMultipler is zero ....

Just being curious - what do values "0 -1" do?
How does it behave if it set to the recommended value of 1.25?
Does it go over the given limit?
How should it help here?
Sorry, I misread the previous answer. Well, I don't need it to rise the heap limit. Actually, script failing is desired behaviour. Rising the limit would cause higher rate of AKS killing the container on memory limit.

If you're only using one engine, there's no difference. ....

Is Runtime a "heavy" part? Would recommend rather creating the Runtime first then create a new Engine per request?
Currently, I create an Engine (via CTOR - each has its own Runtime according to your information) per request. I need memories to be separated - to avoid leakage of the data among engines. Is it still safe doing it one or the another way?

About the occupied memory and the Engine.Dispose. Is it recommended to explicitly call the GC on the Engine before the instance gets disposed or something like that? Does the Dispose free all the memory (ArrayBuffer + Heap)?

[simontom]

Ping

[cw-sarvesh]

We are experiencing similar issues. Do we have any updates from Microsoft?

[ClearScriptLib]

Hi Šimon,

Sorry about the delayed response.

Just being curious - what do values "0 -1" do?

Any value ≤ 1.0 means "no heap expansion".

Well, I don't need it to rise the heap limit. Actually, script failing is desired behaviour. Rising the limit would cause higher rate of AKS killing the container on memory limit.

Unfortunately, heap expansion is the only way to prevent V8 from killing the process when the heap fills up. Heap size monitoring alone is vulnerable to sudden spikes that occur between samples. Combining the two is the only way to stop runaway scripts safely.

Is Runtime a "heavy" part?

Yes, a V8Runtime is an independent copy of the V8 JavaScript engine – complete with its own heap manager, JIT compiler, etc. – but it doesn't include a global object or any of the standard JavaScript built-ins. Those are provided by V8ScriptEngine.

So, a V8ScriptEngine is lightweight compared to a V8Runtime, but the V8ScriptEngine constructor actually creates both. This pattern was intended to facilitate the creation of a complete script execution environment in one step.

Would recommend rather creating the Runtime first then create a new Engine per request?

That depends on your concurrency requirements. The thing to remember is that V8 runtimes are single-threaded. If you need to execute scripts concurrently, you need separate runtimes. V8 runtime construction is expensive though, so using pooling could be advantageous. And if you need a clean environment for each script, CreateScriptEngine should be sufficiently lightweight to provide a decent balance.

I need memories to be separated - to avoid leakage of the data among engines. Is it still safe doing it one or the another way?

Even if multiple script engines share a runtime and therefore a heap, each will have its own object tree rooted at its own global object. Object sharing is possible, but it's something the host would have to set up deliberately.

Is it recommended to explicitly call the GC on the Engine before the instance gets disposed or something like that?

Explicit GC is never required. However, GC is a runtime-level operation. If an engine is sharing a runtime with other engines, disposing it may not immediately collect its objects from the runtime's heap. In that situation, explicit GC could be beneficial.

Does the Dispose free all the memory (ArrayBuffer + Heap)?

Again, it depends. If the engine is using a private runtime, it'll be disposed along with the engine. As for array buffers, that also depends. JavaScript supports SharedArrayBuffer, which allows array buffer memory to be shared across runtimes. Under the covers, each array buffer memory block is reference-counted and released only when its reference count transitions from 1 to zero.

Cheers!