Updated the files.

This commit is contained in:
Batuhan Berk Başoğlu 2024-02-08 19:38:41 -05:00
parent 1553e6b971
commit 753967d4f5
23418 changed files with 3784666 additions and 0 deletions

13
my-app/node_modules/piscina/.taprc generated vendored Executable file
View file

@ -0,0 +1,13 @@
check-coverage: false
color: true
coverage: true
coverage-report:
- html
- text
jobs: 2
no-browser: true
test-env: TS_NODE_PROJECT=test/tsconfig.json
test-ignore: $.
test-regex: ((\/|^)(tests?|__tests?__)\/.*|\.(tests?|spec)|^\/?tests?)\.([mc]js|ts)$
timeout: 60
ts: true

24
my-app/node_modules/piscina/CHANGELOG.md generated vendored Executable file
View file

@ -0,0 +1,24 @@
# Changelog
All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
### [4.2.1](https://github.com/piscinajs/piscina/compare/v4.2.0...v4.2.1) (2023-12-13)
### Bug Fixes
* default minThreads with odd CPU count ([#457](https://github.com/piscinajs/piscina/issues/457)) ([f4edf87](https://github.com/piscinajs/piscina/commit/f4edf87c8c4883e06ab70e99a8a5050eded89c5d))
## [4.2.0](https://github.com/piscinajs/piscina/compare/v4.1.0...v4.2.0) (2023-11-19)
### Features
* Add `Piscina#close` API ([#396](https://github.com/piscinajs/piscina/issues/396)) ([5378e4c](https://github.com/piscinajs/piscina/commit/5378e4cf9143587d9457d3cef6b88aa9653749bd))
### Bug Fixes
* add signal reason support ([#403](https://github.com/piscinajs/piscina/issues/403)) ([66809f9](https://github.com/piscinajs/piscina/commit/66809f94868b4b4597401e10252e1285fabc63c2))
* do not re-create threads when calling `.destory()` ([#430](https://github.com/piscinajs/piscina/issues/430)) ([ec21ff2](https://github.com/piscinajs/piscina/commit/ec21ff28f90a4d5e001ba694fe3dcd6abec3f553))
* migrate to EventEmitterAsyncResource from core ([#433](https://github.com/piscinajs/piscina/issues/433)) ([0a539e2](https://github.com/piscinajs/piscina/commit/0a539e23e7c413cc33631f1adb32ab28b468297b))

129
my-app/node_modules/piscina/CODE_OF_CONDUCT.md generated vendored Executable file
View file

@ -0,0 +1,129 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
jasnell@gmail.com, anna@addaleax.net, or matteo.collina@gmail.com.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series
of actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within
the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.

27
my-app/node_modules/piscina/CONTRIBUTING generated vendored Executable file
View file

@ -0,0 +1,27 @@
# Piscina is an OPEN Open Source Project
## What?
Individuals making significant and valuable contributions are given commit-access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project.
## Rules
There are a few basic ground-rules for contributors:
1. **No `--force` pushes** on `master` or modifying the Git history in any way after a PR has been merged.
1. **Non-master branches** ought to be used for ongoing work.
1. **External API changes and significant modifications** ought to be subject to an **internal pull-request** to solicit feedback from other contributors.
1. Internal pull-requests to solicit feedback are *encouraged* for any other non-trivial contribution but left to the discretion of the contributor.
1. Contributors should attempt to adhere to the prevailing code-style.
1. 100% code coverage
1. Semantic Versioning is used.
## Releases
Declaring formal releases remains the prerogative of the project maintainer.
## Changes to this arrangement
This document may also be subject to pull-requests or changes by contributors where you believe you have something valuable to add or change.
-----------------------------------------

24
my-app/node_modules/piscina/LICENSE generated vendored Executable file
View file

@ -0,0 +1,24 @@
The MIT License (MIT)
Copyright (c) 2020 James M Snell and the Piscina contributors
Piscina contributors listed at https://github.com/jasnell/piscina#the-team and
in the README file.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

954
my-app/node_modules/piscina/README.md generated vendored Executable file
View file

@ -0,0 +1,954 @@
![Piscina Logo](https://avatars1.githubusercontent.com/u/65627548?s=200&v=4)
# piscina - the node.js worker pool
![CI](https://github.com/jasnell/piscina/workflows/CI/badge.svg)
* ✔ Fast communication between threads
* ✔ Covers both fixed-task and variable-task scenarios
* ✔ Supports flexible pool sizes
* ✔ Proper async tracking integration
* ✔ Tracking statistics for run and wait times
* ✔ Cancellation Support
* ✔ Supports enforcing memory resource limits
* ✔ Supports CommonJS, ESM, and TypeScript
* ✔ Custom task queues
* ✔ Optional CPU scheduling priorities on Linux
Written in TypeScript.
For Node.js 16.x and higher.
[MIT Licensed][].
## Piscina API
### Example
In `main.js`:
```js
const path = require('path');
const Piscina = require('piscina');
const piscina = new Piscina({
filename: path.resolve(__dirname, 'worker.js')
});
(async function() {
const result = await piscina.run({ a: 4, b: 6 });
console.log(result); // Prints 10
})();
```
In `worker.js`:
```js
module.exports = ({ a, b }) => {
return a + b;
};
```
The worker may also be an async function or may return a Promise:
```js
const { setTimeout } = require('timers/promises');
module.exports = async ({ a, b }) => {
// Fake some async activity
await setTimeout(100);
return a + b;
};
```
ESM is also supported for both Piscina and workers:
```js
import { Piscina } from 'piscina';
const piscina = new Piscina({
// The URL must be a file:// URL
filename: new URL('./worker.mjs', import.meta.url).href
});
const result = await piscina.run({ a: 4, b: 6 });
console.log(result); // Prints 10
```
In `worker.mjs`:
```js
export default ({ a, b }) => {
return a + b;
};
```
### Exporting multiple worker functions
A single worker file may export multiple named handler functions.
```js
'use strict';
function add({ a, b }) { return a + b; }
function multiply({ a, b }) { return a * b; }
add.add = add;
add.multiply = multiply;
module.exports = add;
```
The export to target can then be specified when the task is submitted:
```js
'use strict';
const Piscina = require('piscina');
const { resolve } = require('path');
const piscina = new Piscina({
filename: resolve(__dirname, 'worker.js')
});
(async function() {
const res = await Promise.all([
piscina.run({ a: 4, b: 6 }, { name: 'add' }),
piscina.run({ a: 4, b: 6 }, { name: 'multiply' })
]);
})();
```
### Cancelable Tasks
Submitted tasks may be canceled using either an `AbortController` or
an `EventEmitter`:
```js
'use strict';
const Piscina = require('piscina');
const { AbortController } = require('abort-controller');
const { resolve } = require('path');
const piscina = new Piscina({
filename: resolve(__dirname, 'worker.js')
});
(async function() {
const abortController = new AbortController();
try {
const { signal } = abortController;
const task = piscina.run({ a: 4, b: 6 }, { signal });
abortController.abort();
await task;
} catch (err) {
console.log('The task was canceled');
}
})();
```
To use `AbortController`, you will need to `npm i abort-controller`
(or `yarn add abort-controller`).
(In Node.js 15.0.0 or higher, there is a new built-in `AbortController`
implementation that can be used here as well.)
Alternatively, any `EventEmitter` that emits an `'abort'` event
may be used as an abort controller:
```js
'use strict';
const Piscina = require('piscina');
const EventEmitter = require('events');
const { resolve } = require('path');
const piscina = new Piscina({
filename: resolve(__dirname, 'worker.js')
});
(async function() {
const ee = new EventEmitter();
try {
const task = piscina.run({ a: 4, b: 6 }, { signal: ee });
ee.emit('abort');
await task;
} catch (err) {
console.log('The task was canceled');
}
})();
```
### Delaying Availability of Workers
A worker thread will not be made available to process tasks until Piscina
determines that it is "ready". By default, a worker is ready as soon as
Piscina loads it and acquires a reference to the exported handler function.
There may be times when the availability of a worker may need to be delayed
longer while the worker initializes any resources it may need to operate.
To support this case, the worker module may export a `Promise` that resolves
the handler function as opposed to exporting the function directly:
```js
async function initialize() {
await someAsyncInitializationActivity();
return ({ a, b }) => a + b;
}
module.exports = initialize();
```
Piscina will await the resolution of the exported Promise before marking
the worker thread available.
### Backpressure
When the `maxQueue` option is set, once the `Piscina` queue is full, no
additional tasks may be submitted until the queue size falls below the
limit. The `'drain'` event may be used to receive notification when the
queue is empty and all tasks have been submitted to workers for processing.
Example: Using a Node.js stream to feed a Piscina worker pool:
```js
'use strict';
const { resolve } = require('path');
const Pool = require('../..');
const pool = new Pool({
filename: resolve(__dirname, 'worker.js'),
maxQueue: 'auto'
});
const stream = getStreamSomehow();
stream.setEncoding('utf8');
pool.on('drain', () => {
if (stream.isPaused()) {
console.log('resuming...', counter, pool.queueSize);
stream.resume();
}
});
stream
.on('data', (data) => {
pool.run(data);
if (pool.queueSize === pool.options.maxQueue) {
console.log('pausing...', counter, pool.queueSize);
stream.pause();
}
})
.on('error', console.error)
.on('end', () => {
console.log('done');
});
```
### Out of scope asynchronous code
A worker thread is **only** active until the moment it returns a result, it can be a result of a synchronous call or a Promise that will be fulfilled/rejected in the future. Once this is done, Piscina will wait for stdout and stderr to be flushed, and then pause the worker's event-loop until the next call. If async code is scheduled without being awaited before returning since Piscina has no way of detecting this, that code execution will be resumed on the next call. Thus, it is highly recommended to properly handle all async tasks before returning a result as it could make your code unpredictable.
For example:
```js
const { setTimeout } = require('timers/promises');
module.exports = ({ a, b }) => {
// This promise should be awaited
setTimeout(1000).then(() => {
console.log('Working'); // This will **not** run during the same worker call
});
return a + b;
};
```
### Additional Examples
Additional examples can be found in the GitHub repo at
https://github.com/piscinajs/piscina/tree/master/examples
## Class: `Piscina`
Piscina works by creating a pool of Node.js Worker Threads to which
one or more tasks may be dispatched. Each worker thread executes a
single exported function defined in a separate file. Whenever a
task is dispatched to a worker, the worker invokes the exported
function and reports the return value back to Piscina when the
function completes.
This class extends [`EventEmitter`][] from Node.js.
### Constructor: `new Piscina([options])`
* The following optional configuration is supported:
* `filename`: (`string | null`) Provides the default source for the code that
runs the tasks on Worker threads. This should be an absolute path or an
absolute `file://` URL to a file that exports a JavaScript `function` or
`async function` as its default export or `module.exports`. [ES modules][]
are supported.
* `name`: (`string | null`) Provides the name of the default exported worker
function. The default is `'default'`, indicating the default export of the
worker module.
* `minThreads`: (`number`) Sets the minimum number of threads that are always
running for this thread pool. The default is based on the number of
available CPUs.
* `maxThreads`: (`number`) Sets the maximum number of threads that are
running for this thread pool. The default is based on the number of
available CPUs.
* `idleTimeout`: (`number`) A timeout in milliseconds that specifies how long
a `Worker` is allowed to be idle, i.e. not handling any tasks, before it is
shut down. By default, this is immediate. **Tip**: *The default `idleTimeout`
can lead to some performance loss in the application because of the overhead
involved with stopping and starting new worker threads. To improve performance,
try setting the `idleTimeout` explicitly.*
* `maxQueue`: (`number` | `string`) The maximum number of tasks that may be
scheduled to run, but not yet running due to lack of available threads, at
a given time. By default, there is no limit. The special value `'auto'`
may be used to have Piscina calculate the maximum as the square of `maxThreads`.
When `'auto'` is used, the calculated `maxQueue` value may be found by checking
the [`options.maxQueue`](#property-options-readonly) property.
* `concurrentTasksPerWorker`: (`number`) Specifies how many tasks can share
a single Worker thread simultaneously. The default is `1`. This generally
only makes sense to specify if there is some kind of asynchronous component
to the task. Keep in mind that Worker threads are generally not built for
handling I/O in parallel.
* `useAtomics`: (`boolean`) Use the [`Atomics`][] API for faster communication
between threads. This is on by default. You can disable `Atomics` globally by
setting the environment variable `PISCINA_DISABLE_ATOMICS` to `1`.
If `useAtomics` is `true`, it will cause to pause threads (stoping all execution)
between tasks. Ideally, threads should wait for all operations to finish before
returning control to the main thread (avoid having open handles within a thread).
* `resourceLimits`: (`object`) See [Node.js new Worker options][]
* `maxOldGenerationSizeMb`: (`number`) The maximum size of each worker threads
main heap in MB.
* `maxYoungGenerationSizeMb`: (`number`) The maximum size of a heap space for
recently created objects.
* `codeRangeSizeMb`: (`number`) The size of a pre-allocated memory range used
for generated code.
* `stackSizeMb` : (`number`) The default maximum stack size for the thread.
Small values may lead to unusable Worker instances. Default: 4
* `env`: (`object`) If set, specifies the initial value of `process.env` inside
the worker threads. See [Node.js new Worker options][] for details.
* `argv`: (`any[]`) List of arguments that will be stringified and appended to
`process.argv` in the worker. See [Node.js new Worker options][] for details.
* `execArgv`: (`string[]`) List of Node.js CLI options passed to the worker.
See [Node.js new Worker options][] for details.
* `workerData`: (`any`) Any JavaScript value that can be cloned and made
available as `require('piscina').workerData`. See [Node.js new Worker options][]
for details. Unlike regular Node.js Worker Threads, `workerData` must not
specify any value requiring a `transferList`. This is because the `workerData`
will be cloned for each pooled worker.
* `taskQueue`: (`TaskQueue`) By default, Piscina uses a first-in-first-out
queue for submitted tasks. The `taskQueue` option can be used to provide an
alternative implementation. See [Custom Task Queues][] for additional detail.
* `niceIncrement`: (`number`) An optional value that decreases priority for
the individual threads, i.e. the higher the value, the lower the priority
of the Worker threads. This value is only used on Linux and requires the
optional [`nice-napi`][] module to be installed.
See [`nice(2)`][] for more details.
* `trackUnmanagedFds`: (`boolean`) An optional setting that, when `true`, will
cause Workers to track file descriptors managed using `fs.open()` and
`fs.close()`, and will close them automatically when the Worker exits.
Defaults to `true`. (This option is only supported on Node.js 12.19+ and
all Node.js versions higher than 14.6.0).
* `closeTimeout`: (`number`) An optional time (in milliseconds) to wait for the pool to
complete all in-flight tasks when `close()` is called. The default is `30000`
Use caution when setting resource limits. Setting limits that are too low may
result in the `Piscina` worker threads being unusable.
### Method: `run(task[, options])`
Schedules a task to be run on a Worker thread.
* `task`: Any value. This will be passed to the function that is exported from
`filename`.
* `options`:
* `transferList`: An optional lists of objects that is passed to
[`postMessage()`] when posting `task` to the Worker, which are transferred
rather than cloned.
* `filename`: Optionally overrides the `filename` option passed to the
constructor for this task. If no `filename` was specified to the constructor,
this is mandatory.
* `name`: Optionally overrides the exported worker function used for the task.
* `abortSignal`: An [`AbortSignal`][] instance. If passed, this can be used to
cancel a task. If the task is already running, the corresponding `Worker`
thread will be stopped.
(More generally, any `EventEmitter` or `EventTarget` that emits `'abort'`
events can be passed here.) Abortable tasks cannot share threads regardless
of the `concurrentTasksPerWorker` options.
This returns a `Promise` for the return value of the (async) function call
made to the function exported from `filename`. If the (async) function throws
an error, the returned `Promise` will be rejected with that error.
If the task is aborted, the returned `Promise` is rejected with an error
as well.
### Method: `runTask(task[, transferList][, filename][, abortSignal])`
**Deprecated** -- Use `run(task, options)` instead.
Schedules a task to be run on a Worker thread.
* `task`: Any value. This will be passed to the function that is exported from
`filename`.
* `transferList`: An optional lists of objects that is passed to
[`postMessage()`] when posting `task` to the Worker, which are transferred
rather than cloned.
* `filename`: Optionally overrides the `filename` option passed to the
constructor for this task. If no `filename` was specified to the constructor,
this is mandatory.
* `abortSignal`: An [`AbortSignal`][] instance. If passed, this can be used to
cancel a task. If the task is already running, the corresponding `Worker`
thread will be stopped.
(More generally, any `EventEmitter` or `EventTarget` that emits `'abort'`
events can be passed here.) Abortable tasks cannot share threads regardless
of the `concurrentTasksPerWorker` options.
This returns a `Promise` for the return value of the (async) function call
made to the function exported from `filename`. If the (async) function throws
an error, the returned `Promise` will be rejected with that error.
If the task is aborted, the returned `Promise` is rejected with an error
as well.
### Method: `destroy()`
Stops all Workers and rejects all `Promise`s for pending tasks.
This returns a `Promise` that is fulfilled once all threads have stopped.
### Method: `close([options])`
* `options`:
* `force`: A `boolean` value that indicates whether to abort all tasks that
are enqueued but not started yet. The default is `false`.
Stops all Workers gracefully.
This returns a `Promise` that is fulfilled once all tasks that were started
have completed and all threads have stopped.
This method is similar to `destroy()`, but with the difference that `close()`
will wait for the worker tasks to finish, while `destroy()`
will abort them immediately.
### Event: `'error'`
An `'error'` event is emitted by instances of this class when:
- Uncaught exceptions occur inside Worker threads that do not currently handle
tasks.
- Unexpected messages are sent from from Worker threads.
All other errors are reported by rejecting the `Promise` returned from
`run()` or `runTask()`, including rejections reported by the handler function
itself.
### Event: `'drain'`
A `'drain'` event is emitted whenever the `queueSize` reaches `0`.
### Event: `'needsDrain'`
Similar to [`Piscina#needsDrain`](#property-needsdrain-readonly);
this event is triggered once the total capacity of the pool is exceeded
by number of tasks enqueued that are pending of execution.
### Event: `'message'`
A `'message'` event is emitted whenever a message is received from a worker thread.
### Property: `completed` (readonly)
The current number of completed tasks.
### Property: `duration` (readonly)
The length of time (in milliseconds) since this `Piscina` instance was
created.
### Property: `options` (readonly)
A copy of the options that are currently being used by this instance. This
object has the same properties as the options object passed to the constructor.
### Property: `runTime` (readonly)
A histogram summary object summarizing the collected run times of completed
tasks. All values are expressed in milliseconds.
* `runTime.average` {`number`} The average run time of all tasks
* `runTime.mean` {`number`} The mean run time of all tasks
* `runTime.stddev` {`number`} The standard deviation of collected run times
* `runTime.min` {`number`} The fastest recorded run time
* `runTime.max` {`number`} The slowest recorded run time
All properties following the pattern `p{N}` where N is a number (e.g. `p1`, `p99`)
represent the percentile distributions of run time observations. For example,
`p99` is the 99th percentile indicating that 99% of the observed run times were
faster or equal to the given value.
```js
{
average: 1880.25,
mean: 1880.25,
stddev: 1.93,
min: 1877,
max: 1882.0190887451172,
p0_001: 1877,
p0_01: 1877,
p0_1: 1877,
p1: 1877,
p2_5: 1877,
p10: 1877,
p25: 1877,
p50: 1881,
p75: 1881,
p90: 1882,
p97_5: 1882,
p99: 1882,
p99_9: 1882,
p99_99: 1882,
p99_999: 1882
}
```
### Property: `threads` (readonly)
An Array of the `Worker` instances used by this pool.
### Property: `queueSize` (readonly)
The current number of tasks waiting to be assigned to a Worker thread.
### Property: `needsDrain` (readonly)
Boolean value that specifies whether the capacity of the pool has
been exceeded by the number of tasks submitted.
This property is helpful to make decisions towards creating backpressure
over the number of tasks submitted to the pool.
### Property: `utilization` (readonly)
A point-in-time ratio comparing the approximate total mean run time
of completed tasks to the total runtime capacity of the pool.
A pools runtime capacity is determined by multiplying the `duration`
by the `options.maxThread` count. This provides an absolute theoretical
maximum aggregate compute time that the pool would be capable of.
The approximate total mean run time is determined by multiplying the
mean run time of all completed tasks by the total number of completed
tasks. This number represents the approximate amount of time the
pool as been actively processing tasks.
The utilization is then calculated by dividing the approximate total
mean run time by the capacity, yielding a fraction between `0` and `1`.
### Property: `waitTime` (readonly)
A histogram summary object summarizing the collected times tasks spent
waiting in the queue. All values are expressed in milliseconds.
* `waitTime.average` {`number`} The average wait time of all tasks
* `waitTime.mean` {`number`} The mean wait time of all tasks
* `waitTime.stddev` {`number`} The standard deviation of collected wait times
* `waitTime.min` {`number`} The fastest recorded wait time
* `waitTime.max` {`number`} The longest recorded wait time
All properties following the pattern `p{N}` where N is a number (e.g. `p1`, `p99`)
represent the percentile distributions of wait time observations. For example,
`p99` is the 99th percentile indicating that 99% of the observed wait times were
faster or equal to the given value.
```js
{
average: 1880.25,
mean: 1880.25,
stddev: 1.93,
min: 1877,
max: 1882.0190887451172,
p0_001: 1877,
p0_01: 1877,
p0_1: 1877,
p1: 1877,
p2_5: 1877,
p10: 1877,
p25: 1877,
p50: 1881,
p75: 1881,
p90: 1882,
p97_5: 1882,
p99: 1882,
p99_9: 1882,
p99_99: 1882,
p99_999: 1882
}
```
### Static property: `isWorkerThread` (readonly)
Is `true` if this code runs inside a `Piscina` threadpool as a Worker.
### Static property: `version` (readonly)
Provides the current version of this library as a semver string.
### Static method: `move(value)`
By default, any value returned by a worker function will be cloned when
returned back to the Piscina pool, even if that object is capable of
being transfered. The `Piscina.move()` method can be used to wrap and
mark transferable values such that they will by transfered rather than
cloned.
The `value` may be any object supported by Node.js to be transferable
(e.g. `ArrayBuffer`, any `TypedArray`, or `MessagePort`), or any object
implementing the `Transferable` interface.
```js
const { move } = require('piscina');
module.exports = () => {
return move(new ArrayBuffer(10));
}
```
The `move()` method will throw if the `value` is not transferable.
The object returned by the `move()` method should not be set as a
nested value in an object. If it is used, the `move()` object itself
will be cloned as opposed to transfering the object it wraps.
#### Interface: `Transferable`
Objects may implement the `Transferable` interface to create their own
custom transferable objects. This is useful when an object being
passed into or from a worker contains a deeply nested transferable
object such as an `ArrayBuffer` or `MessagePort`.
`Transferable` objects expose two properties inspected by Piscina
to determine how to transfer the object. These properties are
named using the special static `Piscina.transferableSymbol` and
`Piscina.valueSymbol` properties:
* The `Piscina.transferableSymbol` property provides the object
(or objects) that are to be included in the `transferList`.
* The `Piscina.valueSymbol` property provides a surrogate value
to transmit in place of the `Transferable` itself.
Both properties are required.
For example,
```js
const {
move,
transferableSymbol,
valueSymbol
} = require('piscina');
module.exports = () => {
const obj = {
a: { b: new Uint8Array(5); },
c: { new Uint8Array(10); },
get [transferableSymbol]() {
// Transfer the two underlying ArrayBuffers
return [this.a.b.buffer, this.c.buffer];
}
get [valueSymbol]() {
return { a: { b: this.a.b }, c: this.c };
}
};
return move(obj);
};
```
## Custom Task Queues
By default, Piscina uses a simple array-based first-in-first-out (fifo)
task queue. When a new task is submitted and there are no available
workers, tasks are pushed on to the queue until a worker becomes
available.
If the default fifo queue is not sufficient, user code may replace the
task queue implementation with a custom implementation using the
`taskQueue` option on the Piscina constructor.
Custom task queue objects *must* implement the `TaskQueue` interface,
described below using TypeScript syntax:
```ts
interface Task {
readonly [Piscina.queueOptionsSymbol] : object | null;
}
interface TaskQueue {
readonly size : number;
shift () : Task | null;
remove (task : Task) : void;
push (task : Task) : void;
}
```
An example of a custom task queue that uses a shuffled priority queue
is available in [`examples/task-queue`](./examples/task-queue/index.js);
The special symbol `Piscina.queueOptionsSymbol` may be set as a property
on tasks submitted to `run()` or `runTask()` as a way of passing additional
options on to the custom `TaskQueue` implementation. (Note that because the
queue options are set as a property on the task, tasks with queue
options cannot be submitted as JavaScript primitives).
## Current Limitations (Things we're working on / would love help with)
* Improved Documentation
* Benchmarks
## Performance Notes
Workers are generally optimized for offloading synchronous,
compute-intensive operations off the main Node.js event loop thread.
While it is possible to perform asynchronous operations and I/O
within a Worker, the performance advantages of doing so will be
minimal.
Specifically, it is worth noting that asynchronous operations
within Node.js, including I/O such as file system operations
or CPU-bound tasks such as crypto operations or compression
algorithms, are already performed in parallel by Node.js and
libuv on a per-process level. This means that there will be
little performance impact on moving such async operations into
a Piscina worker (see examples/scrypt for example).
### Queue Size
Piscina provides the ability to configure the minimum and
maximum number of worker threads active in the pool, as well as
set limits on the number of tasks that may be queued up waiting
for a free worker. It is important to note that setting the
`maxQueue` size too high relative to the number of worker threads
can have a detrimental impact on performance and memory usage.
Setting the `maxQueue` size too small can also be problematic
as doing so could cause your worker threads to become idle and
be shutdown. Our testing has shown that a `maxQueue` size of
approximately the square of the maximum number of threads is
generally sufficient and performs well for many cases, but this
will vary significantly depending on your workload. It will be
important to test and benchmark your worker pools to ensure you've
effectively balanced queue wait times, memory usage, and worker
pool utilization.
### Queue Pressure and Idle Threads
The thread pool maintained by Piscina has both a minimum and maximum
limit to the number of threads that may be created. When a Piscina
instance is created, it will spawn the minimum number of threads
immediately, then create additional threads as needed up to the
limit set by `maxThreads`. Whenever a worker completes a task, a
check is made to determine if there is additional work for it to
perform. If there is no additional work, the thread is marked idle.
By default, idle threads are shutdown immediately, with Piscina
ensuring that the pool always maintains at least the minimum.
When a Piscina pool is processing a stream of tasks (for instance,
processing http server requests as in the React server-side
rendering example in examples/react-ssr), if the rate in which
new tasks are received and queued is not sufficient to keep workers
from going idle and terminating, the pool can experience a thrashing
effect -- excessively creating and terminating workers that will
cause a net performance loss. There are a couple of strategies to
avoid this churn:
Strategy 1: Ensure that the queue rate of new tasks is sufficient to
keep workers from going idle. We refer to this as "queue pressure".
If the queue pressure is too low, workers will go idle and terminate.
If the queue pressure is too high, tasks will stack up, experience
increased wait latency, and consume additional memory.
Strategy 2: Increase the `idleTimeout` configuration option. By
default, idle threads terminate immediately. The `idleTimeout` option
can be used to specify a longer period of time to wait for additional
tasks to be submitted before terminating the worker. If the queue
pressure is not maintained, this could result in workers sitting idle
but those will have less of a performance impact than the thrashing
that occurs when threads are repeatedly terminated and recreated.
Strategy 3: Increase the `minThreads` configuration option. This has
the same basic effect as increasing the `idleTimeout`. If the queue
pressure is not high enough, workers may sit idle indefinitely but
there will be less of a performance hit.
In applications using Piscina, it will be most effective to use a
combination of these three approaches and tune the various configuration
parameters to find the optimum combination both for the application
workload and the capabilities of the deployment environment. There
are no one set of options that are going to work best.
### Thread priority on Linux systems
On Linux systems that support [`nice(2)`][], Piscina is capable of setting
the priority of every worker in the pool. To use this mechanism, an additional
optional native addon dependency (`nice-napi`, `npm i nice-napi`) is required.
Once [`nice-napi`][] is installed, creating a `Piscina` instance with the
`niceIncrement` configuration option will set the priority for the pool:
```js
const Piscina = require('piscina');
const pool = new Piscina({
worker: '/absolute/path/to/worker.js',
niceIncrement: 20
});
```
The higher the `niceIncrement`, the lower the CPU scheduling priority will be
for the pooled workers which will generally extend the execution time of
CPU-bound tasks but will help prevent those threads from stealing CPU time from
the main Node.js event loop thread. Whether this is a good thing or not depends
entirely on your application and will require careful profiling to get correct.
The key metrics to pay attention to when tuning the `niceIncrement` are the
sampled run times of the tasks in the worker pool (using the [`runTime`][]
property) and the [delay of the Node.js main thread event loop][].
### Multiple Thread Pools and Embedding Piscina as a Dependency
Every `Piscina` instance creates a separate pool of threads and operates
without any awareness of the other. When multiple pools are created in a
single application the various threads may contend with one another, and
with the Node.js main event loop thread, and may cause an overall reduction
in system performance.
Modules that embed Piscina as a dependency *should* make it clear via
documentation that threads are being used. It would be ideal if those
would make it possible for users to provide an existing `Piscina` instance
as a configuration option in lieu of always creating their own.
## Release Notes
### 4.1.0
#### Features
* add `needsDrain` property ([#368](https://github.com/piscinajs/piscina/issues/368)) ([2d49b63](https://github.com/piscinajs/piscina/commit/2d49b63368116c172a52e2019648049b4d280162))
* correctly handle process.exit calls outside of a task ([#361](https://github.com/piscinajs/piscina/issues/361)) ([8e6d16e](https://github.com/piscinajs/piscina/commit/8e6d16e1dc23f8bb39772ed954f6689852ad435f))
#### Bug Fixes
* Fix types for TypeScript 4.7 ([#239](https://github.com/piscinajs/piscina/issues/239)) ([a38fb29](https://github.com/piscinajs/piscina/commit/a38fb292e8fcc45cc20abab8668f82d908a24dc0))
* use CJS imports ([#374](https://github.com/piscinajs/piscina/issues/374)) ([edf8dc4](https://github.com/piscinajs/piscina/commit/edf8dc4f1a19e9b49e266109cdb70d9acc86f3ca))
### 4.0.0
* Drop Node.js 14.x support
* Add Node.js 20.x to CI
### 3.2.0
* Adds a new `PISCINA_DISABLE_ATOMICS` environment variable as an alternative way of
disabling Piscina's internal use of the `Atomics` API. (https://github.com/piscinajs/piscina/pull/163)
* Fixes a bug with transferable objects. (https://github.com/piscinajs/piscina/pull/155)
* Fixes CI issues with TypeScript. (https://github.com/piscinajs/piscina/pull/161)
### 3.1.0
* Deprecates `piscina.runTask()`; adds `piscina.run()` as an alternative.
https://github.com/piscinajs/piscina/commit/d7fa24d7515789001f7237ad6ae9ad42d582fc75
* Allows multiple exported handler functions from a single file.
https://github.com/piscinajs/piscina/commit/d7fa24d7515789001f7237ad6ae9ad42d582fc75
### 3.0.0
* Drops Node.js 10.x support
* Updates minimum TypeScript target to ES2019
### 2.1.0
* Adds name property to indicate `AbortError` when tasks are
canceled using an `AbortController` (or similar)
* More examples
### 2.0.0
* Added unmanaged file descriptor tracking
* Updated dependencies
### 1.6.1
* Bug fix: Reject if AbortSignal is already aborted
* Bug Fix: Use once listener for abort event
### 1.6.0
* Add the `niceIncrement` configuration parameter.
### 1.5.1
* Bug fixes around abortable task selection.
### 1.5.0
* Added `Piscina.move()`
* Added Custom Task Queues
* Added utilization metric
* Wait for workers to be ready before considering them as candidates
* Additional examples
### 1.4.0
* Added `maxQueue = 'auto'` to autocalculate the maximum queue size.
* Added more examples, including an example of implementing a worker
as a Node.js native addon.
### 1.3.0
* Added the `'drain'` event
### 1.2.0
* Added support for ESM and file:// URLs
* Added `env`, `argv`, `execArgv`, and `workerData` options
* More examples
### 1.1.0
* Added support for Worker Thread `resourceLimits`
### 1.0.0
* Initial release!
## The Team
* James M Snell <jasnell@gmail.com>
* Anna Henningsen <anna@addaleax.net>
* Matteo Collina <matteo.collina@gmail.com>
## Acknowledgements
Piscina development is sponsored by [NearForm Research][].
[`Atomics`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Atomics
[`EventEmitter`]: https://nodejs.org/api/events.html
[`postMessage`]: https://nodejs.org/api/worker_threads.html#worker_threads_port_postmessage_value_transferlist
[`examples/task-queue`]: https://github.com/jasnell/piscina/blob/master/examples/task-queue/index.js
[`nice(2)`]: https://linux.die.net/man/2/nice
[`nice-napi`]: https://npmjs.org/package/nice-napi
[`runTime`]: #property-runtime-readonly
[Custom Task Queues]: #custom_task_queues
[ES modules]: https://nodejs.org/api/esm.html
[Node.js new Worker options]: https://nodejs.org/api/worker_threads.html#worker_threads_new_worker_filename_options
[MIT Licensed]: LICENSE.md
[NearForm Research]: https://www.nearform.com/research/
[delay of the Node.js main thread event loop]: https://nodejs.org/dist/latest-v14.x/docs/api/perf_hooks.html#perf_hooks_perf_hooks_monitoreventloopdelay_options

2
my-app/node_modules/piscina/benchmark/fixtures/add.js generated vendored Executable file
View file

@ -0,0 +1,2 @@
'use strict';
module.exports = ({ a, b }) => a + b;

29
my-app/node_modules/piscina/benchmark/simple-benchmark.js generated vendored Executable file
View file

@ -0,0 +1,29 @@
'use strict';
const { Piscina } = require('..');
const { resolve } = require('path');
async function simpleBenchmark ({ duration = 10000 } = {}) {
const pool = new Piscina({ filename: resolve(__dirname, 'fixtures/add.js') });
let done = 0;
const results = [];
const start = process.hrtime.bigint();
while (pool.queueSize === 0) {
results.push(scheduleTasks());
}
async function scheduleTasks () {
while ((process.hrtime.bigint() - start) / 1_000_000n < duration) {
await pool.runTask({ a: 4, b: 6 });
done++;
}
}
await Promise.all(results);
return done / duration * 1e3;
}
simpleBenchmark().then((opsPerSecond) => {
console.log(`opsPerSecond: ${opsPerSecond}`);
});

11
my-app/node_modules/piscina/dist/esm-wrapper.mjs generated vendored Executable file
View file

@ -0,0 +1,11 @@
import mod from "./src/index.js";
export default mod;
export const Piscina = mod.Piscina;
export const isWorkerThread = mod.isWorkerThread;
export const move = mod.move;
export const queueOptionsSymbol = mod.queueOptionsSymbol;
export const transferableSymbol = mod.transferableSymbol;
export const valueSymbol = mod.valueSymbol;
export const version = mod.version;
export const workerData = mod.workerData;

89
my-app/node_modules/piscina/dist/package.json generated vendored Executable file
View file

@ -0,0 +1,89 @@
{
"name": "piscina",
"version": "4.2.1",
"description": "A fast, efficient Node.js Worker Thread Pool implementation",
"main": "./dist/src/index.js",
"exports": {
"types": "./dist/src/index.d.ts",
"import": "./dist/esm-wrapper.mjs",
"require": "./dist/src/index.js"
},
"types": "./dist/src/index.d.ts",
"scripts": {
"build": "tsc && gen-esm-wrapper . dist/esm-wrapper.mjs",
"lint": "standardx \"**/*.{ts,mjs,js,cjs}\" | snazzy",
"test": "tap --ts",
"test:ci": "npm run lint && npm run build && npm run test:coverage",
"test:coverage": "tap --ts --cov --coverage-report=html --no-browser --no-check-coverage",
"prepack": "npm run build"
},
"repository": {
"type": "git",
"url": "git+https://github.com/piscinajs/piscina.git"
},
"keywords": [
"fast",
"worker threads",
"thread pool",
"wade wilson"
],
"author": "James M Snell <jasnell@gmail.com>",
"contributors": [
"Anna Henningsen <anna@addaleax.net>",
"Matteo Collina <matteo.collina@gmail.com>"
],
"license": "MIT",
"devDependencies": {
"@types/node": "^20.8.0",
"@typescript-eslint/eslint-plugin": "^6.9.0",
"@typescript-eslint/parser": "^6.9.0",
"abort-controller": "^3.0.0",
"concat-stream": "^2.0.0",
"gen-esm-wrapper": "^1.1.1",
"snazzy": "^9.0.0",
"standardx": "^7.0.0",
"tap": "^16.3.7",
"ts-node": "^10.9.2",
"typescript": "5.3.2"
},
"dependencies": {
"hdr-histogram-js": "^2.0.1",
"hdr-histogram-percentiles-obj": "^3.0.0"
},
"optionalDependencies": {
"nice-napi": "^1.0.2"
},
"eslintConfig": {
"rules": {
"semi": [
"error",
"always"
],
"no-unused-vars": "off",
"no-use-before-define": "off",
"no-unreachable-loop": "off",
"no-dupe-class-members": "off",
"@typescript-eslint/no-unused-vars": "error"
},
"globals": {
"SharedArrayBuffer": true,
"Atomics": true,
"AbortController": true,
"MessageChannel": true
}
},
"standardx": {
"parser": "@typescript-eslint/parser",
"plugins": [
"@typescript-eslint/eslint-plugin"
]
},
"bugs": {
"url": "https://github.com/piscinajs/piscina/issues"
},
"homepage": "https://github.com/piscinajs/piscina#readme",
"directories": {
"example": "examples",
"test": "test"
}
}

52
my-app/node_modules/piscina/dist/src/common.d.ts generated vendored Executable file
View file

@ -0,0 +1,52 @@
/// <reference types="node" />
import type { MessagePort } from 'worker_threads';
export declare const READY = "_WORKER_READY";
export interface StartupMessage {
filename: string | null;
name: string;
port: MessagePort;
sharedBuffer: Int32Array;
useAtomics: boolean;
niceIncrement: number;
}
export interface RequestMessage {
taskId: number;
task: any;
filename: string;
name: string;
}
export interface ReadyMessage {
[READY]: true;
}
export interface ResponseMessage {
taskId: number;
result: any;
error: Error | null;
}
export declare const commonState: {
isWorkerThread: boolean;
workerData: undefined;
};
export declare const kTransferable: unique symbol;
export declare const kValue: unique symbol;
export declare const kQueueOptions: unique symbol;
export declare function isTransferable(value: any): boolean;
export declare function isMovable(value: any): boolean;
export declare function markMovable(value: object): void;
export interface Transferable {
readonly [kTransferable]: object;
readonly [kValue]: object;
}
export interface Task {
readonly [kQueueOptions]: object | null;
}
export interface TaskQueue {
readonly size: number;
shift(): Task | null;
remove(task: Task): void;
push(task: Task): void;
}
export declare function isTaskQueue(value: any): boolean;
export declare const kRequestCountField = 0;
export declare const kResponseCountField = 1;
export declare const kFieldCount = 2;

51
my-app/node_modules/piscina/dist/src/common.js generated vendored Executable file
View file

@ -0,0 +1,51 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.kFieldCount = exports.kResponseCountField = exports.kRequestCountField = exports.isTaskQueue = exports.markMovable = exports.isMovable = exports.isTransferable = exports.kQueueOptions = exports.kValue = exports.kTransferable = exports.commonState = exports.READY = void 0;
exports.READY = '_WORKER_READY';
;
exports.commonState = {
isWorkerThread: false,
workerData: undefined
};
// Internal symbol used to mark Transferable objects returned
// by the Piscina.move() function
const kMovable = Symbol('Piscina.kMovable');
exports.kTransferable = Symbol.for('Piscina.transferable');
exports.kValue = Symbol.for('Piscina.valueOf');
exports.kQueueOptions = Symbol.for('Piscina.queueOptions');
// True if the object implements the Transferable interface
function isTransferable(value) {
return value != null &&
typeof value === 'object' &&
exports.kTransferable in value &&
exports.kValue in value;
}
exports.isTransferable = isTransferable;
// True if object implements Transferable and has been returned
// by the Piscina.move() function
function isMovable(value) {
return isTransferable(value) && value[kMovable] === true;
}
exports.isMovable = isMovable;
function markMovable(value) {
Object.defineProperty(value, kMovable, {
enumerable: false,
configurable: true,
writable: true,
value: true
});
}
exports.markMovable = markMovable;
function isTaskQueue(value) {
return typeof value === 'object' &&
value !== null &&
'size' in value &&
typeof value.shift === 'function' &&
typeof value.remove === 'function' &&
typeof value.push === 'function';
}
exports.isTaskQueue = isTaskQueue;
exports.kRequestCountField = 0;
exports.kResponseCountField = 1;
exports.kFieldCount = 2;
//# sourceMappingURL=common.js.map

1
my-app/node_modules/piscina/dist/src/common.js.map generated vendored Executable file
View file

@ -0,0 +1 @@
{"version":3,"file":"common.js","sourceRoot":"","sources":["../../src/common.ts"],"names":[],"mappings":";;;AAEa,QAAA,KAAK,GAAG,eAAe,CAAC;AAoBpC,CAAC;AAOW,QAAA,WAAW,GAAG;IACzB,cAAc,EAAE,KAAK;IACrB,UAAU,EAAE,SAAS;CACtB,CAAC;AAEF,6DAA6D;AAC7D,iCAAiC;AACjC,MAAM,QAAQ,GAAG,MAAM,CAAC,kBAAkB,CAAC,CAAC;AAC/B,QAAA,aAAa,GAAG,MAAM,CAAC,GAAG,CAAC,sBAAsB,CAAC,CAAC;AACnD,QAAA,MAAM,GAAG,MAAM,CAAC,GAAG,CAAC,iBAAiB,CAAC,CAAC;AACvC,QAAA,aAAa,GAAG,MAAM,CAAC,GAAG,CAAC,sBAAsB,CAAC,CAAC;AAEhE,2DAA2D;AAC3D,SAAgB,cAAc,CAAE,KAAW;IACzC,OAAO,KAAK,IAAI,IAAI;QACb,OAAO,KAAK,KAAK,QAAQ;QACzB,qBAAa,IAAI,KAAK;QACtB,cAAM,IAAI,KAAK,CAAC;AACzB,CAAC;AALD,wCAKC;AAED,+DAA+D;AAC/D,iCAAiC;AACjC,SAAgB,SAAS,CAAE,KAAW;IACpC,OAAO,cAAc,CAAC,KAAK,CAAC,IAAI,KAAK,CAAC,QAAQ,CAAC,KAAK,IAAI,CAAC;AAC3D,CAAC;AAFD,8BAEC;AAED,SAAgB,WAAW,CAAE,KAAc;IACzC,MAAM,CAAC,cAAc,CAAC,KAAK,EAAE,QAAQ,EAAE;QACrC,UAAU,EAAE,KAAK;QACjB,YAAY,EAAE,IAAI;QAClB,QAAQ,EAAE,IAAI;QACd,KAAK,EAAE,IAAI;KACZ,CAAC,CAAC;AACL,CAAC;AAPD,kCAOC;AAkBD,SAAgB,WAAW,CAAE,KAAW;IACtC,OAAO,OAAO,KAAK,KAAK,QAAQ;QACzB,KAAK,KAAK,IAAI;QACd,MAAM,IAAI,KAAK;QACf,OAAO,KAAK,CAAC,KAAK,KAAK,UAAU;QACjC,OAAO,KAAK,CAAC,MAAM,KAAK,UAAU;QAClC,OAAO,KAAK,CAAC,IAAI,KAAK,UAAU,CAAC;AAC1C,CAAC;AAPD,kCAOC;AAEY,QAAA,kBAAkB,GAAG,CAAC,CAAC;AACvB,QAAA,mBAAmB,GAAG,CAAC,CAAC;AACxB,QAAA,WAAW,GAAG,CAAC,CAAC"}

107
my-app/node_modules/piscina/dist/src/index.d.ts generated vendored Executable file
View file

@ -0,0 +1,107 @@
/// <reference types="node" />
/// <reference types="node" />
import { Worker, MessagePort } from 'worker_threads';
import { EventEmitterAsyncResource } from 'events';
import { Transferable, TaskQueue } from './common';
interface AbortSignalEventTargetAddOptions {
once: boolean;
}
interface AbortSignalEventTarget {
addEventListener: (name: 'abort', listener: () => void, options?: AbortSignalEventTargetAddOptions) => void;
removeEventListener: (name: 'abort', listener: () => void) => void;
aborted?: boolean;
reason?: unknown;
}
interface AbortSignalEventEmitter {
off: (name: 'abort', listener: () => void) => void;
once: (name: 'abort', listener: () => void) => void;
}
type AbortSignalAny = AbortSignalEventTarget | AbortSignalEventEmitter;
type ResourceLimits = Worker extends {
resourceLimits?: infer T;
} ? T : {};
type EnvSpecifier = typeof Worker extends {
new (filename: never, options?: {
env: infer T;
}): Worker;
} ? T : never;
interface Options {
filename?: string | null;
name?: string;
minThreads?: number;
maxThreads?: number;
idleTimeout?: number;
maxQueue?: number | 'auto';
concurrentTasksPerWorker?: number;
useAtomics?: boolean;
resourceLimits?: ResourceLimits;
argv?: string[];
execArgv?: string[];
env?: EnvSpecifier;
workerData?: any;
taskQueue?: TaskQueue;
niceIncrement?: number;
trackUnmanagedFds?: boolean;
closeTimeout?: number;
}
interface FilledOptions extends Options {
filename: string | null;
name: string;
minThreads: number;
maxThreads: number;
idleTimeout: number;
maxQueue: number;
concurrentTasksPerWorker: number;
useAtomics: boolean;
taskQueue: TaskQueue;
niceIncrement: number;
closeTimeout: number;
}
interface RunOptions {
transferList?: TransferList;
filename?: string | null;
signal?: AbortSignalAny | null;
name?: string | null;
}
interface CloseOptions {
force?: boolean;
}
type TransferList = MessagePort extends {
postMessage(value: any, transferList: infer T): any;
} ? T : never;
type TransferListItem = TransferList extends (infer T)[] ? T : never;
declare class Piscina extends EventEmitterAsyncResource {
#private;
constructor(options?: Options);
/** @deprecated Use run(task, options) instead **/
runTask(task: any, transferList?: TransferList, filename?: string, abortSignal?: AbortSignalAny): Promise<any>;
/** @deprecated Use run(task, options) instead **/
runTask(task: any, transferList?: TransferList, filename?: AbortSignalAny, abortSignal?: undefined): Promise<any>;
/** @deprecated Use run(task, options) instead **/
runTask(task: any, transferList?: string, filename?: AbortSignalAny, abortSignal?: undefined): Promise<any>;
/** @deprecated Use run(task, options) instead **/
runTask(task: any, transferList?: AbortSignalAny, filename?: undefined, abortSignal?: undefined): Promise<any>;
run(task: any, options?: RunOptions): Promise<any>;
close(options?: CloseOptions): Promise<void>;
destroy(): Promise<void>;
get maxThreads(): number;
get minThreads(): number;
get options(): FilledOptions;
get threads(): Worker[];
get queueSize(): number;
get completed(): number;
get waitTime(): any;
get runTime(): any;
get utilization(): number;
get duration(): number;
get needsDrain(): boolean;
static get isWorkerThread(): boolean;
static get workerData(): any;
static get version(): string;
static get Piscina(): typeof Piscina;
static move(val: Transferable | TransferListItem | ArrayBufferView | ArrayBuffer | MessagePort): ArrayBuffer | ArrayBufferView | MessagePort | Transferable;
static get transferableSymbol(): symbol;
static get valueSymbol(): symbol;
static get queueOptionsSymbol(): symbol;
}
export = Piscina;

992
my-app/node_modules/piscina/dist/src/index.js generated vendored Executable file
View file

@ -0,0 +1,992 @@
"use strict";
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
var _DirectlyTransferable_value, _ArrayBufferViewTransferable_view, _Piscina_pool;
const worker_threads_1 = require("worker_threads");
const events_1 = require("events");
const async_hooks_1 = require("async_hooks");
const os_1 = require("os");
const url_1 = require("url");
const path_1 = require("path");
const util_1 = require("util");
const assert_1 = __importDefault(require("assert"));
const hdr_histogram_js_1 = require("hdr-histogram-js");
const perf_hooks_1 = require("perf_hooks");
const hdr_histogram_percentiles_obj_1 = __importDefault(require("hdr-histogram-percentiles-obj"));
const common_1 = require("./common");
const package_json_1 = require("../package.json");
const promises_1 = require("timers/promises");
const cpuCount = (() => {
try {
return (0, os_1.cpus)().length;
}
catch {
/* istanbul ignore next */
return 1;
}
})();
;
function onabort(abortSignal, listener) {
if ('addEventListener' in abortSignal) {
abortSignal.addEventListener('abort', listener, { once: true });
}
else {
abortSignal.once('abort', listener);
}
}
class AbortError extends Error {
constructor(reason) {
// TS does not recognizes the cause clause
// @ts-expect-error
super('The task has been aborted', { cause: reason });
}
get name() { return 'AbortError'; }
}
class ArrayTaskQueue {
constructor() {
this.tasks = [];
}
get size() { return this.tasks.length; }
shift() {
return this.tasks.shift();
}
push(task) {
this.tasks.push(task);
}
remove(task) {
const index = this.tasks.indexOf(task);
assert_1.default.notStrictEqual(index, -1);
this.tasks.splice(index, 1);
}
}
const kDefaultOptions = {
filename: null,
name: 'default',
minThreads: Math.max(Math.floor(cpuCount / 2), 1),
maxThreads: cpuCount * 1.5,
idleTimeout: 0,
maxQueue: Infinity,
concurrentTasksPerWorker: 1,
useAtomics: true,
taskQueue: new ArrayTaskQueue(),
niceIncrement: 0,
trackUnmanagedFds: true,
closeTimeout: 30000
};
const kDefaultRunOptions = {
transferList: undefined,
filename: null,
signal: null,
name: null
};
const kDefaultCloseOptions = {
force: false
};
class DirectlyTransferable {
constructor(value) {
_DirectlyTransferable_value.set(this, void 0);
__classPrivateFieldSet(this, _DirectlyTransferable_value, value, "f");
}
get [(_DirectlyTransferable_value = new WeakMap(), common_1.kTransferable)]() { return __classPrivateFieldGet(this, _DirectlyTransferable_value, "f"); }
get [common_1.kValue]() { return __classPrivateFieldGet(this, _DirectlyTransferable_value, "f"); }
}
class ArrayBufferViewTransferable {
constructor(view) {
_ArrayBufferViewTransferable_view.set(this, void 0);
__classPrivateFieldSet(this, _ArrayBufferViewTransferable_view, view, "f");
}
get [(_ArrayBufferViewTransferable_view = new WeakMap(), common_1.kTransferable)]() { return __classPrivateFieldGet(this, _ArrayBufferViewTransferable_view, "f").buffer; }
get [common_1.kValue]() { return __classPrivateFieldGet(this, _ArrayBufferViewTransferable_view, "f"); }
}
let taskIdCounter = 0;
function maybeFileURLToPath(filename) {
return filename.startsWith('file:')
? (0, url_1.fileURLToPath)(new url_1.URL(filename))
: filename;
}
// Extend AsyncResource so that async relations between posting a task and
// receiving its result are visible to diagnostic tools.
class TaskInfo extends async_hooks_1.AsyncResource {
constructor(task, transferList, filename, name, callback, abortSignal, triggerAsyncId) {
super('Piscina.Task', { requireManualDestroy: true, triggerAsyncId });
this.abortListener = null;
this.workerInfo = null;
this.callback = callback;
this.task = task;
this.transferList = transferList;
// If the task is a Transferable returned by
// Piscina.move(), then add it to the transferList
// automatically
if ((0, common_1.isMovable)(task)) {
// This condition should never be hit but typescript
// complains if we dont do the check.
/* istanbul ignore if */
if (this.transferList == null) {
this.transferList = [];
}
this.transferList =
this.transferList.concat(task[common_1.kTransferable]);
this.task = task[common_1.kValue];
}
this.filename = filename;
this.name = name;
this.taskId = taskIdCounter++;
this.abortSignal = abortSignal;
this.created = perf_hooks_1.performance.now();
this.started = 0;
}
releaseTask() {
const ret = this.task;
this.task = null;
return ret;
}
done(err, result) {
this.runInAsyncScope(this.callback, null, err, result);
this.emitDestroy(); // `TaskInfo`s are used only once.
// If an abort signal was used, remove the listener from it when
// done to make sure we do not accidentally leak.
if (this.abortSignal && this.abortListener) {
if ('removeEventListener' in this.abortSignal && this.abortListener) {
this.abortSignal.removeEventListener('abort', this.abortListener);
}
else {
this.abortSignal.off('abort', this.abortListener);
}
}
}
get [common_1.kQueueOptions]() {
return common_1.kQueueOptions in this.task ? this.task[common_1.kQueueOptions] : null;
}
}
class AsynchronouslyCreatedResource {
constructor() {
this.onreadyListeners = [];
}
markAsReady() {
const listeners = this.onreadyListeners;
(0, assert_1.default)(listeners !== null);
this.onreadyListeners = null;
for (const listener of listeners) {
listener();
}
}
isReady() {
return this.onreadyListeners === null;
}
onReady(fn) {
if (this.onreadyListeners === null) {
fn(); // Zalgo is okay here.
return;
}
this.onreadyListeners.push(fn);
}
}
class AsynchronouslyCreatedResourcePool {
constructor(maximumUsage) {
this.pendingItems = new Set();
this.readyItems = new Set();
this.maximumUsage = maximumUsage;
this.onAvailableListeners = [];
}
add(item) {
this.pendingItems.add(item);
item.onReady(() => {
/* istanbul ignore else */
if (this.pendingItems.has(item)) {
this.pendingItems.delete(item);
this.readyItems.add(item);
this.maybeAvailable(item);
}
});
}
delete(item) {
this.pendingItems.delete(item);
this.readyItems.delete(item);
}
findAvailable() {
let minUsage = this.maximumUsage;
let candidate = null;
for (const item of this.readyItems) {
const usage = item.currentUsage();
if (usage === 0)
return item;
if (usage < minUsage) {
candidate = item;
minUsage = usage;
}
}
return candidate;
}
*[Symbol.iterator]() {
yield* this.pendingItems;
yield* this.readyItems;
}
get size() {
return this.pendingItems.size + this.readyItems.size;
}
maybeAvailable(item) {
/* istanbul ignore else */
if (item.currentUsage() < this.maximumUsage) {
for (const listener of this.onAvailableListeners) {
listener(item);
}
}
}
onAvailable(fn) {
this.onAvailableListeners.push(fn);
}
}
const Errors = {
ThreadTermination: () => new Error('Terminating worker thread'),
FilenameNotProvided: () => new Error('filename must be provided to run() or in options object'),
TaskQueueAtLimit: () => new Error('Task queue is at limit'),
NoTaskQueueAvailable: () => new Error('No task queue available and all Workers are busy'),
CloseTimeout: () => new Error('Close operation timed out')
};
class WorkerInfo extends AsynchronouslyCreatedResource {
constructor(worker, port, onMessage) {
super();
this.idleTimeout = null; // eslint-disable-line no-undef
this.lastSeenResponseCount = 0;
this.worker = worker;
this.port = port;
this.port.on('message', (message) => this._handleResponse(message));
this.onMessage = onMessage;
this.taskInfos = new Map();
this.sharedBuffer = new Int32Array(new SharedArrayBuffer(common_1.kFieldCount * Int32Array.BYTES_PER_ELEMENT));
}
destroy() {
this.worker.terminate();
this.port.close();
this.clearIdleTimeout();
for (const taskInfo of this.taskInfos.values()) {
taskInfo.done(Errors.ThreadTermination());
}
this.taskInfos.clear();
}
clearIdleTimeout() {
if (this.idleTimeout !== null) {
clearTimeout(this.idleTimeout);
this.idleTimeout = null;
}
}
ref() {
this.port.ref();
return this;
}
unref() {
// Note: Do not call ref()/unref() on the Worker itself since that may cause
// a hard crash, see https://github.com/nodejs/node/pull/33394.
this.port.unref();
return this;
}
_handleResponse(message) {
this.onMessage(message);
if (this.taskInfos.size === 0) {
// No more tasks running on this Worker means it should not keep the
// process running.
this.unref();
}
}
postTask(taskInfo) {
(0, assert_1.default)(!this.taskInfos.has(taskInfo.taskId));
const message = {
task: taskInfo.releaseTask(),
taskId: taskInfo.taskId,
filename: taskInfo.filename,
name: taskInfo.name
};
try {
this.port.postMessage(message, taskInfo.transferList);
}
catch (err) {
// This would mostly happen if e.g. message contains unserializable data
// or transferList is invalid.
taskInfo.done(err);
return;
}
taskInfo.workerInfo = this;
this.taskInfos.set(taskInfo.taskId, taskInfo);
this.ref();
this.clearIdleTimeout();
// Inform the worker that there are new messages posted, and wake it up
// if it is waiting for one.
Atomics.add(this.sharedBuffer, common_1.kRequestCountField, 1);
Atomics.notify(this.sharedBuffer, common_1.kRequestCountField, 1);
}
processPendingMessages() {
// If we *know* that there are more messages than we have received using
// 'message' events yet, then try to load and handle them synchronously,
// without the need to wait for more expensive events on the event loop.
// This would usually break async tracking, but in our case, we already have
// the extra TaskInfo/AsyncResource layer that rectifies that situation.
const actualResponseCount = Atomics.load(this.sharedBuffer, common_1.kResponseCountField);
if (actualResponseCount !== this.lastSeenResponseCount) {
this.lastSeenResponseCount = actualResponseCount;
let entry;
while ((entry = (0, worker_threads_1.receiveMessageOnPort)(this.port)) !== undefined) {
this._handleResponse(entry.message);
}
}
}
isRunningAbortableTask() {
// If there are abortable tasks, we are running one at most per Worker.
if (this.taskInfos.size !== 1)
return false;
const [[, task]] = this.taskInfos;
return task.abortSignal !== null;
}
currentUsage() {
if (this.isRunningAbortableTask())
return Infinity;
return this.taskInfos.size;
}
}
class ThreadPool {
constructor(publicInterface, options) {
var _a;
this.skipQueue = [];
this.completed = 0;
this.start = perf_hooks_1.performance.now();
this.inProcessPendingMessages = false;
this.startingUp = false;
this.closingUp = false;
this.workerFailsDuringBootstrap = false;
this.destroying = false;
this.publicInterface = publicInterface;
this.taskQueue = options.taskQueue || new ArrayTaskQueue();
this.runTime = (0, hdr_histogram_js_1.build)({ lowestDiscernibleValue: 1 });
this.waitTime = (0, hdr_histogram_js_1.build)({ lowestDiscernibleValue: 1 });
const filename = options.filename ? maybeFileURLToPath(options.filename) : null;
this.options = { ...kDefaultOptions, ...options, filename, maxQueue: 0 };
// The >= and <= could be > and < but this way we get 100 % coverage 🙃
if (options.maxThreads !== undefined &&
this.options.minThreads >= options.maxThreads) {
this.options.minThreads = options.maxThreads;
}
if (options.minThreads !== undefined &&
this.options.maxThreads <= options.minThreads) {
this.options.maxThreads = options.minThreads;
}
if (options.maxQueue === 'auto') {
this.options.maxQueue = this.options.maxThreads ** 2;
}
else {
this.options.maxQueue = (_a = options.maxQueue) !== null && _a !== void 0 ? _a : kDefaultOptions.maxQueue;
}
this.workers = new AsynchronouslyCreatedResourcePool(this.options.concurrentTasksPerWorker);
this.workers.onAvailable((w) => this._onWorkerAvailable(w));
this.startingUp = true;
this._ensureMinimumWorkers();
this.startingUp = false;
this.needsDrain = false;
}
_ensureMinimumWorkers() {
if (this.closingUp || this.destroying) {
return;
}
while (this.workers.size < this.options.minThreads) {
this._addNewWorker();
}
}
_addNewWorker() {
const pool = this;
const worker = new worker_threads_1.Worker((0, path_1.resolve)(__dirname, 'worker.js'), {
env: this.options.env,
argv: this.options.argv,
execArgv: this.options.execArgv,
resourceLimits: this.options.resourceLimits,
workerData: this.options.workerData,
trackUnmanagedFds: this.options.trackUnmanagedFds
});
const { port1, port2 } = new worker_threads_1.MessageChannel();
const workerInfo = new WorkerInfo(worker, port1, onMessage);
if (this.startingUp) {
// There is no point in waiting for the initial set of Workers to indicate
// that they are ready, we just mark them as such from the start.
workerInfo.markAsReady();
}
const message = {
filename: this.options.filename,
name: this.options.name,
port: port2,
sharedBuffer: workerInfo.sharedBuffer,
useAtomics: this.options.useAtomics,
niceIncrement: this.options.niceIncrement
};
worker.postMessage(message, [port2]);
function onMessage(message) {
const { taskId, result } = message;
// In case of success: Call the callback that was passed to `runTask`,
// remove the `TaskInfo` associated with the Worker, which marks it as
// free again.
const taskInfo = workerInfo.taskInfos.get(taskId);
workerInfo.taskInfos.delete(taskId);
pool.workers.maybeAvailable(workerInfo);
/* istanbul ignore if */
if (taskInfo === undefined) {
const err = new Error(`Unexpected message from Worker: ${(0, util_1.inspect)(message)}`);
pool.publicInterface.emit('error', err);
}
else {
taskInfo.done(message.error, result);
}
pool._processPendingMessages();
}
function onReady() {
if (workerInfo.currentUsage() === 0) {
workerInfo.unref();
}
if (!workerInfo.isReady()) {
workerInfo.markAsReady();
}
}
function onEventMessage(message) {
pool.publicInterface.emit('message', message);
}
worker.on('message', (message) => {
message instanceof Object && common_1.READY in message ? onReady() : onEventMessage(message);
});
worker.on('error', (err) => {
this._onError(worker, workerInfo, err, false);
});
worker.on('exit', (exitCode) => {
if (this.destroying) {
return;
}
const err = new Error(`worker exited with code: ${exitCode}`);
// Only error unfinished tasks on process exit, since there are legitimate
// reasons to exit workers and we want to handle that gracefully when possible.
this._onError(worker, workerInfo, err, true);
});
worker.unref();
port1.on('close', () => {
// The port is only closed if the Worker stops for some reason, but we
// always .unref() the Worker itself. We want to receive e.g. 'error'
// events on it, so we ref it once we know it's going to exit anyway.
worker.ref();
});
this.workers.add(workerInfo);
}
_onError(worker, workerInfo, err, onlyErrorUnfinishedTasks) {
// Work around the bug in https://github.com/nodejs/node/pull/33394
worker.ref = () => { };
const taskInfos = [...workerInfo.taskInfos.values()];
workerInfo.taskInfos.clear();
// Remove the worker from the list and potentially start a new Worker to
// replace the current one.
this._removeWorker(workerInfo);
if (workerInfo.isReady() && !this.workerFailsDuringBootstrap) {
this._ensureMinimumWorkers();
}
else {
// Do not start new workers over and over if they already fail during
// bootstrap, there's no point.
this.workerFailsDuringBootstrap = true;
}
if (taskInfos.length > 0) {
// If there are remaining unfinished tasks, call the callback that was
// passed to `postTask` with the error
for (const taskInfo of taskInfos) {
taskInfo.done(err, null);
}
}
else if (!onlyErrorUnfinishedTasks) {
// If there are no unfinished tasks, instead emit an 'error' event
this.publicInterface.emit('error', err);
}
}
_processPendingMessages() {
if (this.inProcessPendingMessages || !this.options.useAtomics) {
return;
}
this.inProcessPendingMessages = true;
try {
for (const workerInfo of this.workers) {
workerInfo.processPendingMessages();
}
}
finally {
this.inProcessPendingMessages = false;
}
}
_removeWorker(workerInfo) {
workerInfo.destroy();
this.workers.delete(workerInfo);
}
_onWorkerAvailable(workerInfo) {
while ((this.taskQueue.size > 0 || this.skipQueue.length > 0) &&
workerInfo.currentUsage() < this.options.concurrentTasksPerWorker) {
// The skipQueue will have tasks that we previously shifted off
// the task queue but had to skip over... we have to make sure
// we drain that before we drain the taskQueue.
const taskInfo = this.skipQueue.shift() ||
this.taskQueue.shift();
// If the task has an abortSignal and the worker has any other
// tasks, we cannot distribute the task to it. Skip for now.
if (taskInfo.abortSignal && workerInfo.taskInfos.size > 0) {
this.skipQueue.push(taskInfo);
break;
}
const now = perf_hooks_1.performance.now();
this.waitTime.recordValue(now - taskInfo.created);
taskInfo.started = now;
workerInfo.postTask(taskInfo);
this._maybeDrain();
return;
}
if (workerInfo.taskInfos.size === 0 &&
this.workers.size > this.options.minThreads) {
workerInfo.idleTimeout = setTimeout(() => {
assert_1.default.strictEqual(workerInfo.taskInfos.size, 0);
if (this.workers.size > this.options.minThreads) {
this._removeWorker(workerInfo);
}
}, this.options.idleTimeout).unref();
}
}
runTask(task, options) {
var _a;
let { filename, name } = options;
const { transferList = [] } = options;
if (filename == null) {
filename = this.options.filename;
}
if (name == null) {
name = this.options.name;
}
if (typeof filename !== 'string') {
return Promise.reject(Errors.FilenameNotProvided());
}
filename = maybeFileURLToPath(filename);
let signal;
if (this.closingUp) {
const closingUpAbortController = new AbortController();
closingUpAbortController.abort('queue is closing up');
signal = closingUpAbortController.signal;
}
else {
signal = (_a = options.signal) !== null && _a !== void 0 ? _a : null;
}
let resolve;
let reject;
// eslint-disable-next-line
const ret = new Promise((res, rej) => { resolve = res; reject = rej; });
const taskInfo = new TaskInfo(task, transferList, filename, name, (err, result) => {
this.completed++;
if (taskInfo.started) {
this.runTime.recordValue(perf_hooks_1.performance.now() - taskInfo.started);
}
if (err !== null) {
reject(err);
}
else {
resolve(result);
}
this._maybeDrain();
}, signal, this.publicInterface.asyncResource.asyncId());
if (signal !== null) {
// If the AbortSignal has an aborted property and it's truthy,
// reject immediately.
if (signal.aborted) {
return Promise.reject(new AbortError(signal.reason));
}
taskInfo.abortListener = () => {
// Call reject() first to make sure we always reject with the AbortError
// if the task is aborted, not with an Error from the possible
// thread termination below.
reject(new AbortError(signal.reason));
if (taskInfo.workerInfo !== null) {
// Already running: We cancel the Worker this is running on.
this._removeWorker(taskInfo.workerInfo);
this._ensureMinimumWorkers();
}
else {
// Not yet running: Remove it from the queue.
this.taskQueue.remove(taskInfo);
}
};
onabort(signal, taskInfo.abortListener);
}
// If there is a task queue, there's no point in looking for an available
// Worker thread. Add this task to the queue, if possible.
if (this.taskQueue.size > 0) {
const totalCapacity = this.options.maxQueue + this.pendingCapacity();
if (this.taskQueue.size >= totalCapacity) {
if (this.options.maxQueue === 0) {
return Promise.reject(Errors.NoTaskQueueAvailable());
}
else {
return Promise.reject(Errors.TaskQueueAtLimit());
}
}
else {
if (this.workers.size < this.options.maxThreads) {
this._addNewWorker();
}
this.taskQueue.push(taskInfo);
}
this._maybeDrain();
return ret;
}
// Look for a Worker with a minimum number of tasks it is currently running.
let workerInfo = this.workers.findAvailable();
// If we want the ability to abort this task, use only workers that have
// no running tasks.
if (workerInfo !== null && workerInfo.currentUsage() > 0 && signal) {
workerInfo = null;
}
// If no Worker was found, or that Worker was handling another task in some
// way, and we still have the ability to spawn new threads, do so.
let waitingForNewWorker = false;
if ((workerInfo === null || workerInfo.currentUsage() > 0) &&
this.workers.size < this.options.maxThreads) {
this._addNewWorker();
waitingForNewWorker = true;
}
// If no Worker is found, try to put the task into the queue.
if (workerInfo === null) {
if (this.options.maxQueue <= 0 && !waitingForNewWorker) {
return Promise.reject(Errors.NoTaskQueueAvailable());
}
else {
this.taskQueue.push(taskInfo);
}
this._maybeDrain();
return ret;
}
// TODO(addaleax): Clean up the waitTime/runTime recording.
const now = perf_hooks_1.performance.now();
this.waitTime.recordValue(now - taskInfo.created);
taskInfo.started = now;
workerInfo.postTask(taskInfo);
this._maybeDrain();
return ret;
}
pendingCapacity() {
return this.workers.pendingItems.size *
this.options.concurrentTasksPerWorker;
}
_maybeDrain() {
const totalCapacity = this.options.maxQueue + this.pendingCapacity();
const totalQueueSize = this.taskQueue.size + this.skipQueue.length;
if (totalQueueSize === 0) {
this.needsDrain = false;
this.publicInterface.emit('drain');
}
if (totalQueueSize >= totalCapacity) {
this.needsDrain = true;
this.publicInterface.emit('needsDrain');
}
}
async destroy() {
this.destroying = true;
while (this.skipQueue.length > 0) {
const taskInfo = this.skipQueue.shift();
taskInfo.done(new Error('Terminating worker thread'));
}
while (this.taskQueue.size > 0) {
const taskInfo = this.taskQueue.shift();
taskInfo.done(new Error('Terminating worker thread'));
}
const exitEvents = [];
while (this.workers.size > 0) {
const [workerInfo] = this.workers;
exitEvents.push((0, events_1.once)(workerInfo.worker, 'exit'));
this._removeWorker(workerInfo);
}
try {
await Promise.all(exitEvents);
}
finally {
this.destroying = false;
}
}
async close(options) {
this.closingUp = true;
if (options.force) {
const skipQueueLength = this.skipQueue.length;
for (let i = 0; i < skipQueueLength; i++) {
const taskInfo = this.skipQueue.shift();
if (taskInfo.workerInfo === null) {
taskInfo.done(new AbortError('pool is closed'));
}
else {
this.skipQueue.push(taskInfo);
}
}
const taskQueueLength = this.taskQueue.size;
for (let i = 0; i < taskQueueLength; i++) {
const taskInfo = this.taskQueue.shift();
if (taskInfo.workerInfo === null) {
taskInfo.done(new AbortError('pool is closed'));
}
else {
this.taskQueue.push(taskInfo);
}
}
}
const onPoolFlushed = () => new Promise((resolve) => {
const numberOfWorkers = this.workers.size;
let numberOfWorkersDone = 0;
const checkIfWorkerIsDone = (workerInfo) => {
if (workerInfo.taskInfos.size === 0) {
numberOfWorkersDone++;
}
if (numberOfWorkers === numberOfWorkersDone) {
resolve();
}
};
for (const workerInfo of this.workers) {
checkIfWorkerIsDone(workerInfo);
workerInfo.port.on('message', () => checkIfWorkerIsDone(workerInfo));
}
});
const throwOnTimeOut = async (timeout) => {
await (0, promises_1.setTimeout)(timeout);
throw Errors.CloseTimeout();
};
try {
await Promise.race([
onPoolFlushed(),
throwOnTimeOut(this.options.closeTimeout)
]);
}
catch (error) {
this.publicInterface.emit('error', error);
}
finally {
await this.destroy();
this.publicInterface.emit('close');
this.closingUp = false;
}
}
}
class Piscina extends events_1.EventEmitterAsyncResource {
constructor(options = {}) {
super({ ...options, name: 'Piscina' });
_Piscina_pool.set(this, void 0);
if (typeof options.filename !== 'string' && options.filename != null) {
throw new TypeError('options.filename must be a string or null');
}
if (typeof options.name !== 'string' && options.name != null) {
throw new TypeError('options.name must be a string or null');
}
if (options.minThreads !== undefined &&
(typeof options.minThreads !== 'number' || options.minThreads < 0)) {
throw new TypeError('options.minThreads must be a non-negative integer');
}
if (options.maxThreads !== undefined &&
(typeof options.maxThreads !== 'number' || options.maxThreads < 1)) {
throw new TypeError('options.maxThreads must be a positive integer');
}
if (options.minThreads !== undefined && options.maxThreads !== undefined &&
options.minThreads > options.maxThreads) {
throw new RangeError('options.minThreads and options.maxThreads must not conflict');
}
if (options.idleTimeout !== undefined &&
(typeof options.idleTimeout !== 'number' || options.idleTimeout < 0)) {
throw new TypeError('options.idleTimeout must be a non-negative integer');
}
if (options.maxQueue !== undefined &&
options.maxQueue !== 'auto' &&
(typeof options.maxQueue !== 'number' || options.maxQueue < 0)) {
throw new TypeError('options.maxQueue must be a non-negative integer');
}
if (options.concurrentTasksPerWorker !== undefined &&
(typeof options.concurrentTasksPerWorker !== 'number' ||
options.concurrentTasksPerWorker < 1)) {
throw new TypeError('options.concurrentTasksPerWorker must be a positive integer');
}
if (options.useAtomics !== undefined &&
typeof options.useAtomics !== 'boolean') {
throw new TypeError('options.useAtomics must be a boolean value');
}
if (options.resourceLimits !== undefined &&
(typeof options.resourceLimits !== 'object' ||
options.resourceLimits === null)) {
throw new TypeError('options.resourceLimits must be an object');
}
if (options.taskQueue !== undefined && !(0, common_1.isTaskQueue)(options.taskQueue)) {
throw new TypeError('options.taskQueue must be a TaskQueue object');
}
if (options.niceIncrement !== undefined &&
(typeof options.niceIncrement !== 'number' || options.niceIncrement < 0)) {
throw new TypeError('options.niceIncrement must be a non-negative integer');
}
if (options.trackUnmanagedFds !== undefined &&
typeof options.trackUnmanagedFds !== 'boolean') {
throw new TypeError('options.trackUnmanagedFds must be a boolean value');
}
if (options.closeTimeout !== undefined && (typeof options.closeTimeout !== 'number' || options.closeTimeout < 0)) {
throw new TypeError('options.closeTimeout must be a non-negative integer');
}
__classPrivateFieldSet(this, _Piscina_pool, new ThreadPool(this, options), "f");
}
/** @deprecated Use run(task, options) instead **/
runTask(task, transferList, filename, signal) {
// If transferList is a string or AbortSignal, shift it.
if ((typeof transferList === 'object' && !Array.isArray(transferList)) ||
typeof transferList === 'string') {
signal = filename;
filename = transferList;
transferList = undefined;
}
// If filename is an AbortSignal, shift it.
if (typeof filename === 'object' && !Array.isArray(filename)) {
signal = filename;
filename = undefined;
}
if (transferList !== undefined && !Array.isArray(transferList)) {
return Promise.reject(new TypeError('transferList argument must be an Array'));
}
if (filename !== undefined && typeof filename !== 'string') {
return Promise.reject(new TypeError('filename argument must be a string'));
}
if (signal !== undefined && typeof signal !== 'object') {
return Promise.reject(new TypeError('signal argument must be an object'));
}
return __classPrivateFieldGet(this, _Piscina_pool, "f").runTask(task, {
transferList,
filename: filename || null,
name: 'default',
signal: signal || null
});
}
run(task, options = kDefaultRunOptions) {
if (options === null || typeof options !== 'object') {
return Promise.reject(new TypeError('options must be an object'));
}
const { transferList, filename, name, signal } = options;
if (transferList !== undefined && !Array.isArray(transferList)) {
return Promise.reject(new TypeError('transferList argument must be an Array'));
}
if (filename != null && typeof filename !== 'string') {
return Promise.reject(new TypeError('filename argument must be a string'));
}
if (name != null && typeof name !== 'string') {
return Promise.reject(new TypeError('name argument must be a string'));
}
if (signal != null && typeof signal !== 'object') {
return Promise.reject(new TypeError('signal argument must be an object'));
}
return __classPrivateFieldGet(this, _Piscina_pool, "f").runTask(task, { transferList, filename, name, signal });
}
async close(options = kDefaultCloseOptions) {
if (options === null || typeof options !== 'object') {
throw TypeError('options must be an object');
}
let { force } = options;
if (force !== undefined && typeof force !== 'boolean') {
return Promise.reject(new TypeError('force argument must be a boolean'));
}
force !== null && force !== void 0 ? force : (force = kDefaultCloseOptions.force);
return __classPrivateFieldGet(this, _Piscina_pool, "f").close({
force
});
}
destroy() {
return __classPrivateFieldGet(this, _Piscina_pool, "f").destroy();
}
get maxThreads() {
return __classPrivateFieldGet(this, _Piscina_pool, "f").options.maxThreads;
}
get minThreads() {
return __classPrivateFieldGet(this, _Piscina_pool, "f").options.minThreads;
}
get options() {
return __classPrivateFieldGet(this, _Piscina_pool, "f").options;
}
get threads() {
const ret = [];
for (const workerInfo of __classPrivateFieldGet(this, _Piscina_pool, "f").workers) {
ret.push(workerInfo.worker);
}
return ret;
}
get queueSize() {
const pool = __classPrivateFieldGet(this, _Piscina_pool, "f");
return Math.max(pool.taskQueue.size - pool.pendingCapacity(), 0);
}
get completed() {
return __classPrivateFieldGet(this, _Piscina_pool, "f").completed;
}
get waitTime() {
const result = hdr_histogram_percentiles_obj_1.default.histAsObj(__classPrivateFieldGet(this, _Piscina_pool, "f").waitTime);
return hdr_histogram_percentiles_obj_1.default.addPercentiles(__classPrivateFieldGet(this, _Piscina_pool, "f").waitTime, result);
}
get runTime() {
const result = hdr_histogram_percentiles_obj_1.default.histAsObj(__classPrivateFieldGet(this, _Piscina_pool, "f").runTime);
return hdr_histogram_percentiles_obj_1.default.addPercentiles(__classPrivateFieldGet(this, _Piscina_pool, "f").runTime, result);
}
get utilization() {
// The capacity is the max compute time capacity of the
// pool to this point in time as determined by the length
// of time the pool has been running multiplied by the
// maximum number of threads.
const capacity = this.duration * __classPrivateFieldGet(this, _Piscina_pool, "f").options.maxThreads;
const totalMeanRuntime = __classPrivateFieldGet(this, _Piscina_pool, "f").runTime.mean *
__classPrivateFieldGet(this, _Piscina_pool, "f").runTime.totalCount;
// We calculate the appoximate pool utilization by multiplying
// the mean run time of all tasks by the number of runtime
// samples taken and dividing that by the capacity. The
// theory here is that capacity represents the absolute upper
// limit of compute time this pool could ever attain (but
// never will for a variety of reasons. Multiplying the
// mean run time by the number of tasks sampled yields an
// approximation of the realized compute time. The utilization
// then becomes a point-in-time measure of how active the
// pool is.
return totalMeanRuntime / capacity;
}
get duration() {
return perf_hooks_1.performance.now() - __classPrivateFieldGet(this, _Piscina_pool, "f").start;
}
get needsDrain() {
return __classPrivateFieldGet(this, _Piscina_pool, "f").needsDrain;
}
static get isWorkerThread() {
return common_1.commonState.isWorkerThread;
}
static get workerData() {
return common_1.commonState.workerData;
}
static get version() {
return package_json_1.version;
}
static get Piscina() {
return Piscina;
}
static move(val) {
if (val != null && typeof val === 'object' && typeof val !== 'function') {
if (!(0, common_1.isTransferable)(val)) {
if (util_1.types.isArrayBufferView(val)) {
val = new ArrayBufferViewTransferable(val);
}
else {
val = new DirectlyTransferable(val);
}
}
(0, common_1.markMovable)(val);
}
return val;
}
static get transferableSymbol() { return common_1.kTransferable; }
static get valueSymbol() { return common_1.kValue; }
static get queueOptionsSymbol() { return common_1.kQueueOptions; }
}
_Piscina_pool = new WeakMap();
module.exports = Piscina;
//# sourceMappingURL=index.js.map

1
my-app/node_modules/piscina/dist/src/index.js.map generated vendored Executable file

File diff suppressed because one or more lines are too long

0
my-app/node_modules/piscina/dist/src/typescript.d.ts generated vendored Executable file
View file

2
my-app/node_modules/piscina/dist/src/typescript.js generated vendored Executable file
View file

@ -0,0 +1,2 @@
"use strict";
//# sourceMappingURL=typescript.js.map

1
my-app/node_modules/piscina/dist/src/typescript.js.map generated vendored Executable file
View file

@ -0,0 +1 @@
{"version":3,"file":"typescript.js","sourceRoot":"","sources":["../../src/typescript.ts"],"names":[],"mappings":""}

1
my-app/node_modules/piscina/dist/src/worker.d.ts generated vendored Executable file
View file

@ -0,0 +1 @@
export {};

187
my-app/node_modules/piscina/dist/src/worker.js generated vendored Executable file
View file

@ -0,0 +1,187 @@
"use strict";
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k in mod) if (k !== "default" && Object.prototype.hasOwnProperty.call(mod, k)) __createBinding(result, mod, k);
__setModuleDefault(result, mod);
return result;
};
Object.defineProperty(exports, "__esModule", { value: true });
const worker_threads_1 = require("worker_threads");
const url_1 = require("url");
const common_1 = require("./common");
common_1.commonState.isWorkerThread = true;
common_1.commonState.workerData = worker_threads_1.workerData;
const handlerCache = new Map();
let useAtomics = process.env.PISCINA_DISABLE_ATOMICS !== '1';
// Get `import(x)` as a function that isn't transpiled to `require(x)` by
// TypeScript for dual ESM/CJS support.
// Load this lazily, so that there is no warning about the ESM loader being
// experimental (on Node v12.x) until we actually try to use it.
let importESMCached;
function getImportESM() {
if (importESMCached === undefined) {
// eslint-disable-next-line no-new-func
importESMCached = new Function('specifier', 'return import(specifier)');
}
return importESMCached;
}
// Look up the handler function that we call when a task is posted.
// This is either going to be "the" export from a file, or the default export.
async function getHandler(filename, name) {
let handler = handlerCache.get(`${filename}/${name}`);
if (handler !== undefined) {
return handler;
}
try {
// With our current set of TypeScript options, this is transpiled to
// `require(filename)`.
handler = await Promise.resolve(`${filename}`).then(s => __importStar(require(s)));
if (typeof handler !== 'function') {
handler = await (handler[name]);
}
}
catch { }
if (typeof handler !== 'function') {
handler = await getImportESM()((0, url_1.pathToFileURL)(filename).href);
if (typeof handler !== 'function') {
handler = await (handler[name]);
}
}
if (typeof handler !== 'function') {
return null;
}
// Limit the handler cache size. This should not usually be an issue and is
// only provided for pathological cases.
if (handlerCache.size > 1000) {
const [[key]] = handlerCache;
handlerCache.delete(key);
}
handlerCache.set(`${filename}/${name}`, handler);
return handler;
}
// We should only receive this message once, when the Worker starts. It gives
// us the MessagePort used for receiving tasks, a SharedArrayBuffer for fast
// communication using Atomics, and the name of the default filename for tasks
// (so we can pre-load and cache the handler).
worker_threads_1.parentPort.on('message', (message) => {
useAtomics = process.env.PISCINA_DISABLE_ATOMICS === '1' ? false : message.useAtomics;
const { port, sharedBuffer, filename, name, niceIncrement } = message;
(async function () {
try {
if (niceIncrement !== 0 && process.platform === 'linux') {
// ts-ignore because the dependency is not installed on Windows.
// @ts-ignore
(await Promise.resolve().then(() => __importStar(require('nice-napi')))).default(niceIncrement);
}
}
catch { }
if (filename !== null) {
await getHandler(filename, name);
}
const readyMessage = { [common_1.READY]: true };
worker_threads_1.parentPort.postMessage(readyMessage);
port.on('message', onMessage.bind(null, port, sharedBuffer));
atomicsWaitLoop(port, sharedBuffer);
})().catch(throwInNextTick);
});
let currentTasks = 0;
let lastSeenRequestCount = 0;
function atomicsWaitLoop(port, sharedBuffer) {
if (!useAtomics)
return;
// This function is entered either after receiving the startup message, or
// when we are done with a task. In those situations, the *only* thing we
// expect to happen next is a 'message' on `port`.
// That call would come with the overhead of a C++ → JS boundary crossing,
// including async tracking. So, instead, if there is no task currently
// running, we wait for a signal from the parent thread using Atomics.wait(),
// and read the message from the port instead of generating an event,
// in order to avoid that overhead.
// The one catch is that this stops asynchronous operations that are still
// running from proceeding. Generally, tasks should not spawn asynchronous
// operations without waiting for them to finish, though.
while (currentTasks === 0) {
// Check whether there are new messages by testing whether the current
// number of requests posted by the parent thread matches the number of
// requests received.
Atomics.wait(sharedBuffer, common_1.kRequestCountField, lastSeenRequestCount);
lastSeenRequestCount = Atomics.load(sharedBuffer, common_1.kRequestCountField);
// We have to read messages *after* updating lastSeenRequestCount in order
// to avoid race conditions.
let entry;
while ((entry = (0, worker_threads_1.receiveMessageOnPort)(port)) !== undefined) {
onMessage(port, sharedBuffer, entry.message);
}
}
}
function onMessage(port, sharedBuffer, message) {
currentTasks++;
const { taskId, task, filename, name } = message;
(async function () {
let response;
let transferList = [];
try {
const handler = await getHandler(filename, name);
if (handler === null) {
throw new Error(`No handler function exported from ${filename}`);
}
let result = await handler(task);
if ((0, common_1.isMovable)(result)) {
transferList = transferList.concat(result[common_1.kTransferable]);
result = result[common_1.kValue];
}
response = {
taskId,
result: result,
error: null
};
// If the task used e.g. console.log(), wait for the stream to drain
// before potentially entering the `Atomics.wait()` loop, and before
// returning the result so that messages will always be printed even
// if the process would otherwise be ready to exit.
if (process.stdout.writableLength > 0) {
await new Promise((resolve) => process.stdout.write('', resolve));
}
if (process.stderr.writableLength > 0) {
await new Promise((resolve) => process.stderr.write('', resolve));
}
}
catch (error) {
response = {
taskId,
result: null,
// It may be worth taking a look at the error cloning algorithm we
// use in Node.js core here, it's quite a bit more flexible
error: error instanceof Error ? error : null
};
}
currentTasks--;
// Post the response to the parent thread, and let it know that we have
// an additional message available. If possible, use Atomics.wait()
// to wait for the next message.
port.postMessage(response, transferList);
Atomics.add(sharedBuffer, common_1.kResponseCountField, 1);
atomicsWaitLoop(port, sharedBuffer);
})().catch(throwInNextTick);
}
function throwInNextTick(error) {
process.nextTick(() => { throw error; });
}
//# sourceMappingURL=worker.js.map

1
my-app/node_modules/piscina/dist/src/worker.js.map generated vendored Executable file
View file

@ -0,0 +1 @@
{"version":3,"file":"worker.js","sourceRoot":"","sources":["../../src/worker.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;AAAA,mDAA2F;AAC3F,6BAAoC;AACpC,qCAYkB;AAElB,oBAAW,CAAC,cAAc,GAAG,IAAI,CAAC;AAClC,oBAAW,CAAC,UAAU,GAAG,2BAAU,CAAC;AAEpC,MAAM,YAAY,GAA2B,IAAI,GAAG,EAAE,CAAC;AACvD,IAAI,UAAU,GAAa,OAAO,CAAC,GAAG,CAAC,uBAAuB,KAAK,GAAG,CAAC;AAEvE,yEAAyE;AACzE,uCAAuC;AACvC,2EAA2E;AAC3E,gEAAgE;AAChE,IAAI,eAAkE,CAAC;AACvE,SAAS,YAAY;IACnB,IAAI,eAAe,KAAK,SAAS,EAAE;QACjC,uCAAuC;QACvC,eAAe,GAAG,IAAI,QAAQ,CAAC,WAAW,EAAE,0BAA0B,CAA2B,CAAC;KACnG;IACD,OAAO,eAAe,CAAC;AACzB,CAAC;AAED,mEAAmE;AACnE,8EAA8E;AAC9E,KAAK,UAAU,UAAU,CAAE,QAAiB,EAAE,IAAa;IACzD,IAAI,OAAO,GAAG,YAAY,CAAC,GAAG,CAAC,GAAG,QAAQ,IAAI,IAAI,EAAE,CAAC,CAAC;IACtD,IAAI,OAAO,KAAK,SAAS,EAAE;QACzB,OAAO,OAAO,CAAC;KAChB;IAED,IAAI;QACF,oEAAoE;QACpE,uBAAuB;QACvB,OAAO,GAAG,yBAAa,QAAQ,uCAAC,CAAC;QACjC,IAAI,OAAO,OAAO,KAAK,UAAU,EAAE;YACjC,OAAO,GAAG,MAAM,CAAE,OAAe,CAAC,IAAI,CAAC,CAAC,CAAC;SAC1C;KACF;IAAC,MAAM,GAAE;IACV,IAAI,OAAO,OAAO,KAAK,UAAU,EAAE;QACjC,OAAO,GAAG,MAAM,YAAY,EAAE,CAAC,IAAA,mBAAa,EAAC,QAAQ,CAAC,CAAC,IAAI,CAAC,CAAC;QAC7D,IAAI,OAAO,OAAO,KAAK,UAAU,EAAE;YACjC,OAAO,GAAG,MAAM,CAAE,OAAe,CAAC,IAAI,CAAC,CAAC,CAAC;SAC1C;KACF;IACD,IAAI,OAAO,OAAO,KAAK,UAAU,EAAE;QACjC,OAAO,IAAI,CAAC;KACb;IAED,2EAA2E;IAC3E,wCAAwC;IACxC,IAAI,YAAY,CAAC,IAAI,GAAG,IAAI,EAAE;QAC5B,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,YAAY,CAAC;QAC7B,YAAY,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;KAC1B;IAED,YAAY,CAAC,GAAG,CAAC,GAAG,QAAQ,IAAI,IAAI,EAAE,EAAE,OAAO,CAAC,CAAC;IACjD,OAAO,OAAO,CAAC;AACjB,CAAC;AAED,6EAA6E;AAC7E,4EAA4E;AAC5E,8EAA8E;AAC9E,8CAA8C;AAC9C,2BAAW,CAAC,EAAE,CAAC,SAAS,EAAE,CAAC,OAAwB,EAAE,EAAE;IACrD,UAAU,GAAG,OAAO,CAAC,GAAG,CAAC,uBAAuB,KAAK,GAAG,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,OAAO,CAAC,UAAU,CAAC;IACtF,MAAM,EAAE,IAAI,EAAE,YAAY,EAAE,QAAQ,EAAE,IAAI,EAAE,aAAa,EAAE,GAAG,OAAO,CAAC;IACtE,CAAC,KAAK;QACJ,IAAI;YACF,IAAI,aAAa,KAAK,CAAC,IAAI,OAAO,CAAC,QAAQ,KAAK,OAAO,EAAE;gBACvD,gEAAgE;gBAChE,aAAa;gBACb,CAAC,wDAAa,WAAW,GAAC,CAAC,CAAC,OAAO,CAAC,aAAa,CAAC,CAAC;aACpD;SACF;QAAC,MAAM,GAAE;QAEV,IAAI,QAAQ,KAAK,IAAI,EAAE;YACrB,MAAM,UAAU,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC;SAClC;QAED,MAAM,YAAY,GAAkB,EAAE,CAAC,cAAK,CAAC,EAAE,IAAI,EAAE,CAAC;QACtD,2BAAW,CAAC,WAAW,CAAC,YAAY,CAAC,CAAC;QAEtC,IAAI,CAAC,EAAE,CAAC,SAAS,EAAE,SAAS,CAAC,IAAI,CAAC,IAAI,EAAE,IAAI,EAAE,YAAY,CAAC,CAAC,CAAC;QAC7D,eAAe,CAAC,IAAI,EAAE,YAAY,CAAC,CAAC;IACtC,CAAC,CAAC,EAAE,CAAC,KAAK,CAAC,eAAe,CAAC,CAAC;AAC9B,CAAC,CAAC,CAAC;AAEH,IAAI,YAAY,GAAY,CAAC,CAAC;AAC9B,IAAI,oBAAoB,GAAY,CAAC,CAAC;AACtC,SAAS,eAAe,CAAE,IAAkB,EAAE,YAAyB;IACrE,IAAI,CAAC,UAAU;QAAE,OAAO;IAExB,0EAA0E;IAC1E,yEAAyE;IACzE,kDAAkD;IAClD,0EAA0E;IAC1E,uEAAuE;IACvE,6EAA6E;IAC7E,qEAAqE;IACrE,mCAAmC;IACnC,0EAA0E;IAC1E,0EAA0E;IAC1E,yDAAyD;IACzD,OAAO,YAAY,KAAK,CAAC,EAAE;QACzB,sEAAsE;QACtE,uEAAuE;QACvE,qBAAqB;QACrB,OAAO,CAAC,IAAI,CAAC,YAAY,EAAE,2BAAkB,EAAE,oBAAoB,CAAC,CAAC;QACrE,oBAAoB,GAAG,OAAO,CAAC,IAAI,CAAC,YAAY,EAAE,2BAAkB,CAAC,CAAC;QAEtE,0EAA0E;QAC1E,4BAA4B;QAC5B,IAAI,KAAK,CAAC;QACV,OAAO,CAAC,KAAK,GAAG,IAAA,qCAAoB,EAAC,IAAI,CAAC,CAAC,KAAK,SAAS,EAAE;YACzD,SAAS,CAAC,IAAI,EAAE,YAAY,EAAE,KAAK,CAAC,OAAO,CAAC,CAAC;SAC9C;KACF;AACH,CAAC;AAED,SAAS,SAAS,CAChB,IAAkB,EAClB,YAAyB,EACzB,OAAwB;IACxB,YAAY,EAAE,CAAC;IACf,MAAM,EAAE,MAAM,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,GAAG,OAAO,CAAC;IAEjD,CAAC,KAAK;QACJ,IAAI,QAA0B,CAAC;QAC/B,IAAI,YAAY,GAAW,EAAE,CAAC;QAC9B,IAAI;YACF,MAAM,OAAO,GAAG,MAAM,UAAU,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC;YACjD,IAAI,OAAO,KAAK,IAAI,EAAE;gBACpB,MAAM,IAAI,KAAK,CAAC,qCAAqC,QAAQ,EAAE,CAAC,CAAC;aAClE;YACD,IAAI,MAAM,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;YACjC,IAAI,IAAA,kBAAS,EAAC,MAAM,CAAC,EAAE;gBACrB,YAAY,GAAG,YAAY,CAAC,MAAM,CAAC,MAAM,CAAC,sBAAa,CAAC,CAAC,CAAC;gBAC1D,MAAM,GAAG,MAAM,CAAC,eAAM,CAAC,CAAC;aACzB;YACD,QAAQ,GAAG;gBACT,MAAM;gBACN,MAAM,EAAE,MAAM;gBACd,KAAK,EAAE,IAAI;aACZ,CAAC;YAEF,oEAAoE;YACpE,oEAAoE;YACpE,oEAAoE;YACpE,mDAAmD;YACnD,IAAI,OAAO,CAAC,MAAM,CAAC,cAAc,GAAG,CAAC,EAAE;gBACrC,MAAM,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,EAAE,OAAO,CAAC,CAAC,CAAC;aACnE;YACD,IAAI,OAAO,CAAC,MAAM,CAAC,cAAc,GAAG,CAAC,EAAE;gBACrC,MAAM,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,EAAE,OAAO,CAAC,CAAC,CAAC;aACnE;SACF;QAAC,OAAO,KAAK,EAAE;YACd,QAAQ,GAAG;gBACT,MAAM;gBACN,MAAM,EAAE,IAAI;gBACZ,kEAAkE;gBAClE,2DAA2D;gBAC3D,KAAK,EAAE,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI;aAC7C,CAAC;SACH;QACD,YAAY,EAAE,CAAC;QAEf,uEAAuE;QACvE,mEAAmE;QACnE,gCAAgC;QAChC,IAAI,CAAC,WAAW,CAAC,QAAQ,EAAE,YAAY,CAAC,CAAC;QACzC,OAAO,CAAC,GAAG,CAAC,YAAY,EAAE,4BAAmB,EAAE,CAAC,CAAC,CAAC;QAClD,eAAe,CAAC,IAAI,EAAE,YAAY,CAAC,CAAC;IACtC,CAAC,CAAC,EAAE,CAAC,KAAK,CAAC,eAAe,CAAC,CAAC;AAC9B,CAAC;AAED,SAAS,eAAe,CAAE,KAAa;IACrC,OAAO,CAAC,QAAQ,CAAC,GAAG,EAAE,GAAG,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;AAC3C,CAAC"}

89
my-app/node_modules/piscina/package.json generated vendored Executable file
View file

@ -0,0 +1,89 @@
{
"name": "piscina",
"version": "4.2.1",
"description": "A fast, efficient Node.js Worker Thread Pool implementation",
"main": "./dist/src/index.js",
"exports": {
"types": "./dist/src/index.d.ts",
"import": "./dist/esm-wrapper.mjs",
"require": "./dist/src/index.js"
},
"types": "./dist/src/index.d.ts",
"scripts": {
"build": "tsc && gen-esm-wrapper . dist/esm-wrapper.mjs",
"lint": "standardx \"**/*.{ts,mjs,js,cjs}\" | snazzy",
"test": "tap --ts",
"test:ci": "npm run lint && npm run build && npm run test:coverage",
"test:coverage": "tap --ts --cov --coverage-report=html --no-browser --no-check-coverage",
"prepack": "npm run build"
},
"repository": {
"type": "git",
"url": "git+https://github.com/piscinajs/piscina.git"
},
"keywords": [
"fast",
"worker threads",
"thread pool",
"wade wilson"
],
"author": "James M Snell <jasnell@gmail.com>",
"contributors": [
"Anna Henningsen <anna@addaleax.net>",
"Matteo Collina <matteo.collina@gmail.com>"
],
"license": "MIT",
"devDependencies": {
"@types/node": "^20.8.0",
"@typescript-eslint/eslint-plugin": "^6.9.0",
"@typescript-eslint/parser": "^6.9.0",
"abort-controller": "^3.0.0",
"concat-stream": "^2.0.0",
"gen-esm-wrapper": "^1.1.1",
"snazzy": "^9.0.0",
"standardx": "^7.0.0",
"tap": "^16.3.7",
"ts-node": "^10.9.2",
"typescript": "5.3.2"
},
"dependencies": {
"hdr-histogram-js": "^2.0.1",
"hdr-histogram-percentiles-obj": "^3.0.0"
},
"optionalDependencies": {
"nice-napi": "^1.0.2"
},
"eslintConfig": {
"rules": {
"semi": [
"error",
"always"
],
"no-unused-vars": "off",
"no-use-before-define": "off",
"no-unreachable-loop": "off",
"no-dupe-class-members": "off",
"@typescript-eslint/no-unused-vars": "error"
},
"globals": {
"SharedArrayBuffer": true,
"Atomics": true,
"AbortController": true,
"MessageChannel": true
}
},
"standardx": {
"parser": "@typescript-eslint/parser",
"plugins": [
"@typescript-eslint/eslint-plugin"
]
},
"bugs": {
"url": "https://github.com/piscinajs/piscina/issues"
},
"homepage": "https://github.com/piscinajs/piscina#readme",
"directories": {
"example": "examples",
"test": "test"
}
}

92
my-app/node_modules/piscina/src/common.ts generated vendored Executable file
View file

@ -0,0 +1,92 @@
import type { MessagePort } from 'worker_threads';
export const READY = '_WORKER_READY';
export interface StartupMessage {
filename : string | null;
name : string;
port : MessagePort;
sharedBuffer : Int32Array;
useAtomics : boolean;
niceIncrement : number;
}
export interface RequestMessage {
taskId : number;
task : any;
filename: string;
name : string;
}
export interface ReadyMessage {
[READY]: true
};
export interface ResponseMessage {
taskId : number;
result : any;
error: Error | null;
}
export const commonState = {
isWorkerThread: false,
workerData: undefined
};
// Internal symbol used to mark Transferable objects returned
// by the Piscina.move() function
const kMovable = Symbol('Piscina.kMovable');
export const kTransferable = Symbol.for('Piscina.transferable');
export const kValue = Symbol.for('Piscina.valueOf');
export const kQueueOptions = Symbol.for('Piscina.queueOptions');
// True if the object implements the Transferable interface
export function isTransferable (value : any) : boolean {
return value != null &&
typeof value === 'object' &&
kTransferable in value &&
kValue in value;
}
// True if object implements Transferable and has been returned
// by the Piscina.move() function
export function isMovable (value : any) : boolean {
return isTransferable(value) && value[kMovable] === true;
}
export function markMovable (value : object) : void {
Object.defineProperty(value, kMovable, {
enumerable: false,
configurable: true,
writable: true,
value: true
});
}
export interface Transferable {
readonly [kTransferable] : object;
readonly [kValue] : object;
}
export interface Task {
readonly [kQueueOptions] : object | null;
}
export interface TaskQueue {
readonly size : number;
shift () : Task | null;
remove (task : Task) : void;
push (task : Task) : void;
}
export function isTaskQueue (value : any) : boolean {
return typeof value === 'object' &&
value !== null &&
'size' in value &&
typeof value.shift === 'function' &&
typeof value.remove === 'function' &&
typeof value.push === 'function';
}
export const kRequestCountField = 0;
export const kResponseCountField = 1;
export const kFieldCount = 2;

View file

@ -0,0 +1 @@
declare module 'hdr-histogram-percentiles-obj';

1292
my-app/node_modules/piscina/src/index.ts generated vendored Executable file

File diff suppressed because it is too large Load diff

191
my-app/node_modules/piscina/src/worker.ts generated vendored Executable file
View file

@ -0,0 +1,191 @@
import { parentPort, MessagePort, receiveMessageOnPort, workerData } from 'worker_threads';
import { pathToFileURL } from 'url';
import {
READY,
commonState,
ReadyMessage,
RequestMessage,
ResponseMessage,
StartupMessage,
kResponseCountField,
kRequestCountField,
isMovable,
kTransferable,
kValue
} from './common';
commonState.isWorkerThread = true;
commonState.workerData = workerData;
const handlerCache : Map<string, Function> = new Map();
let useAtomics : boolean = process.env.PISCINA_DISABLE_ATOMICS !== '1';
// Get `import(x)` as a function that isn't transpiled to `require(x)` by
// TypeScript for dual ESM/CJS support.
// Load this lazily, so that there is no warning about the ESM loader being
// experimental (on Node v12.x) until we actually try to use it.
let importESMCached : (specifier : string) => Promise<any> | undefined;
function getImportESM () {
if (importESMCached === undefined) {
// eslint-disable-next-line no-new-func
importESMCached = new Function('specifier', 'return import(specifier)') as typeof importESMCached;
}
return importESMCached;
}
// Look up the handler function that we call when a task is posted.
// This is either going to be "the" export from a file, or the default export.
async function getHandler (filename : string, name : string) : Promise<Function | null> {
let handler = handlerCache.get(`${filename}/${name}`);
if (handler !== undefined) {
return handler;
}
try {
// With our current set of TypeScript options, this is transpiled to
// `require(filename)`.
handler = await import(filename);
if (typeof handler !== 'function') {
handler = await ((handler as any)[name]);
}
} catch {}
if (typeof handler !== 'function') {
handler = await getImportESM()(pathToFileURL(filename).href);
if (typeof handler !== 'function') {
handler = await ((handler as any)[name]);
}
}
if (typeof handler !== 'function') {
return null;
}
// Limit the handler cache size. This should not usually be an issue and is
// only provided for pathological cases.
if (handlerCache.size > 1000) {
const [[key]] = handlerCache;
handlerCache.delete(key);
}
handlerCache.set(`${filename}/${name}`, handler);
return handler;
}
// We should only receive this message once, when the Worker starts. It gives
// us the MessagePort used for receiving tasks, a SharedArrayBuffer for fast
// communication using Atomics, and the name of the default filename for tasks
// (so we can pre-load and cache the handler).
parentPort!.on('message', (message : StartupMessage) => {
useAtomics = process.env.PISCINA_DISABLE_ATOMICS === '1' ? false : message.useAtomics;
const { port, sharedBuffer, filename, name, niceIncrement } = message;
(async function () {
try {
if (niceIncrement !== 0 && process.platform === 'linux') {
// ts-ignore because the dependency is not installed on Windows.
// @ts-ignore
(await import('nice-napi')).default(niceIncrement);
}
} catch {}
if (filename !== null) {
await getHandler(filename, name);
}
const readyMessage : ReadyMessage = { [READY]: true };
parentPort!.postMessage(readyMessage);
port.on('message', onMessage.bind(null, port, sharedBuffer));
atomicsWaitLoop(port, sharedBuffer);
})().catch(throwInNextTick);
});
let currentTasks : number = 0;
let lastSeenRequestCount : number = 0;
function atomicsWaitLoop (port : MessagePort, sharedBuffer : Int32Array) {
if (!useAtomics) return;
// This function is entered either after receiving the startup message, or
// when we are done with a task. In those situations, the *only* thing we
// expect to happen next is a 'message' on `port`.
// That call would come with the overhead of a C++ → JS boundary crossing,
// including async tracking. So, instead, if there is no task currently
// running, we wait for a signal from the parent thread using Atomics.wait(),
// and read the message from the port instead of generating an event,
// in order to avoid that overhead.
// The one catch is that this stops asynchronous operations that are still
// running from proceeding. Generally, tasks should not spawn asynchronous
// operations without waiting for them to finish, though.
while (currentTasks === 0) {
// Check whether there are new messages by testing whether the current
// number of requests posted by the parent thread matches the number of
// requests received.
Atomics.wait(sharedBuffer, kRequestCountField, lastSeenRequestCount);
lastSeenRequestCount = Atomics.load(sharedBuffer, kRequestCountField);
// We have to read messages *after* updating lastSeenRequestCount in order
// to avoid race conditions.
let entry;
while ((entry = receiveMessageOnPort(port)) !== undefined) {
onMessage(port, sharedBuffer, entry.message);
}
}
}
function onMessage (
port : MessagePort,
sharedBuffer : Int32Array,
message : RequestMessage) {
currentTasks++;
const { taskId, task, filename, name } = message;
(async function () {
let response : ResponseMessage;
let transferList : any[] = [];
try {
const handler = await getHandler(filename, name);
if (handler === null) {
throw new Error(`No handler function exported from ${filename}`);
}
let result = await handler(task);
if (isMovable(result)) {
transferList = transferList.concat(result[kTransferable]);
result = result[kValue];
}
response = {
taskId,
result: result,
error: null
};
// If the task used e.g. console.log(), wait for the stream to drain
// before potentially entering the `Atomics.wait()` loop, and before
// returning the result so that messages will always be printed even
// if the process would otherwise be ready to exit.
if (process.stdout.writableLength > 0) {
await new Promise((resolve) => process.stdout.write('', resolve));
}
if (process.stderr.writableLength > 0) {
await new Promise((resolve) => process.stderr.write('', resolve));
}
} catch (error) {
response = {
taskId,
result: null,
// It may be worth taking a look at the error cloning algorithm we
// use in Node.js core here, it's quite a bit more flexible
error: error instanceof Error ? error : null
};
}
currentTasks--;
// Post the response to the parent thread, and let it know that we have
// an additional message available. If possible, use Atomics.wait()
// to wait for the next message.
port.postMessage(response, transferList);
Atomics.add(sharedBuffer, kResponseCountField, 1);
atomicsWaitLoop(port, sharedBuffer);
})().catch(throwInNextTick);
}
function throwInNextTick (error : Error) {
process.nextTick(() => { throw error; });
}

233
my-app/node_modules/piscina/test/abort-task.ts generated vendored Executable file
View file

@ -0,0 +1,233 @@
import { EventEmitter } from 'events';
import Piscina from '..';
import { test } from 'tap';
import { resolve } from 'path';
test('tasks can be aborted through AbortController while running', async ({ equal, rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/notify-then-sleep.ts')
});
const buf = new Int32Array(new SharedArrayBuffer(4));
const abortController = new AbortController();
rejects(pool.runTask(buf, abortController.signal),
/The task has been aborted/);
Atomics.wait(buf, 0, 0);
equal(Atomics.load(buf, 0), 1);
abortController.abort();
});
test('tasks can be aborted through EventEmitter while running', async ({ equal, rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/notify-then-sleep.ts')
});
const buf = new Int32Array(new SharedArrayBuffer(4));
const ee = new EventEmitter();
rejects(pool.runTask(buf, ee), /The task has been aborted/);
rejects(pool.run(buf, { signal: ee }), /The task has been aborted/);
Atomics.wait(buf, 0, 0);
equal(Atomics.load(buf, 0), 1);
ee.emit('abort');
});
test('tasks can be aborted through EventEmitter before running', async ({ equal, rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/wait-for-notify.ts'),
maxThreads: 1
});
const bufs = [
new Int32Array(new SharedArrayBuffer(4)),
new Int32Array(new SharedArrayBuffer(4))
];
const task1 = pool.runTask(bufs[0]);
const ee = new EventEmitter();
rejects(pool.runTask(bufs[1], ee), /The task has been aborted/);
rejects(pool.run(bufs[1], { signal: ee }), /The task has been aborted/);
equal(pool.queueSize, 2);
ee.emit('abort');
// Wake up the thread handling the first task.
Atomics.store(bufs[0], 0, 1);
Atomics.notify(bufs[0], 0, 1);
await task1;
});
test('abortable tasks will not share workers (abortable posted second)', async ({ equal, rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/wait-for-notify.ts'),
maxThreads: 1,
concurrentTasksPerWorker: 2
});
const bufs = [
new Int32Array(new SharedArrayBuffer(4)),
new Int32Array(new SharedArrayBuffer(4))
];
const task1 = pool.runTask(bufs[0]);
const ee = new EventEmitter();
rejects(pool.runTask(bufs[1], ee), /The task has been aborted/);
equal(pool.queueSize, 1);
ee.emit('abort');
// Wake up the thread handling the first task.
Atomics.store(bufs[0], 0, 1);
Atomics.notify(bufs[0], 0, 1);
await task1;
});
test('abortable tasks will not share workers (abortable posted first)', async ({ equal, rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
maxThreads: 1,
concurrentTasksPerWorker: 2
});
const ee = new EventEmitter();
rejects(pool.runTask('while(true);', ee), /The task has been aborted/);
const task2 = pool.runTask('42');
equal(pool.queueSize, 1);
ee.emit('abort');
// Wake up the thread handling the second task.
equal(await task2, 42);
});
test('abortable tasks will not share workers (on worker available)', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/sleep.js'),
maxThreads: 1,
concurrentTasksPerWorker: 2
});
// Task 1 will sleep 100 ms then complete,
// Task 2 will sleep 300 ms then complete.
// Abortable task 3 should still be in the queue
// when Task 1 completes, but should not be selected
// until after Task 2 completes because it is abortable.
const ret = await Promise.all([
pool.runTask({ time: 100, a: 1 }),
pool.runTask({ time: 300, a: 2 }),
pool.runTask({ time: 100, a: 3 }, new EventEmitter())
]);
equal(ret[0], 0);
equal(ret[1], 1);
equal(ret[2], 2);
});
test('abortable tasks will not share workers (destroy workers)', async ({ rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/sleep.js'),
maxThreads: 1,
concurrentTasksPerWorker: 2
});
// Task 1 will sleep 100 ms then complete,
// Task 2 will sleep 300 ms then complete.
// Abortable task 3 should still be in the queue
// when Task 1 completes, but should not be selected
// until after Task 2 completes because it is abortable.
pool.runTask({ time: 100, a: 1 }).then(() => {
pool.destroy();
});
rejects(pool.runTask({ time: 300, a: 2 }), /Terminating worker thread/);
rejects(pool.runTask({ time: 100, a: 3 }, new EventEmitter()),
/Terminating worker thread/);
});
test('aborted AbortSignal rejects task immediately', async ({ rejects, equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/move.ts')
});
const controller = new AbortController();
// Abort the controller early
controller.abort();
equal(controller.signal.aborted, true);
// The data won't be moved because the task will abort immediately.
const data = new Uint8Array(new SharedArrayBuffer(4));
rejects(pool.runTask(data, [data.buffer], controller.signal),
/The task has been aborted/);
equal(data.length, 4);
});
test('task with AbortSignal cleans up properly', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
const ee = new EventEmitter();
await pool.runTask('1+1', ee);
const { getEventListeners } = EventEmitter as any;
if (typeof getEventListeners === 'function') {
equal(getEventListeners(ee, 'abort').length, 0);
}
const controller = new AbortController();
await pool.runTask('1+1', controller.signal);
});
test('aborted AbortSignal rejects task immediately (with reason)', async ({ match, equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/move.ts')
});
const customReason = new Error('custom reason');
const controller = new AbortController();
controller.abort(customReason);
equal(controller.signal.aborted, true);
equal(controller.signal.reason, customReason);
// The data won't be moved because the task will abort immediately.
const data = new Uint8Array(new SharedArrayBuffer(4));
try {
await pool.run(data, { transferList: [data.buffer], signal: controller.signal });
} catch (error) {
equal(error.message, 'The task has been aborted');
match(error.cause, customReason);
}
equal(data.length, 4);
});
test('tasks can be aborted through AbortController while running', async ({ equal, match }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/notify-then-sleep.ts')
});
const reason = new Error('custom reason');
const buf = new Int32Array(new SharedArrayBuffer(4));
const abortController = new AbortController();
try {
const promise = pool.run(buf, { signal: abortController.signal });
Atomics.wait(buf, 0, 0);
equal(Atomics.load(buf, 0), 1);
abortController.abort(reason);
await promise;
} catch (error) {
equal(error.message, 'The task has been aborted');
match(error.cause, reason);
}
});

43
my-app/node_modules/piscina/test/async-context.ts generated vendored Executable file
View file

@ -0,0 +1,43 @@
import { createHook, executionAsyncId } from 'async_hooks';
import Piscina from '..';
import { test } from 'tap';
import { resolve } from 'path';
test('postTask() calls the correct async hooks', async ({ equal }) => {
let taskId;
let initCalls = 0;
let beforeCalls = 0;
let afterCalls = 0;
let resolveCalls = 0;
const hook = createHook({
init (id, type) {
if (type === 'Piscina.Task') {
initCalls++;
taskId = id;
}
},
before (id) {
if (id === taskId) beforeCalls++;
},
after (id) {
if (id === taskId) afterCalls++;
},
promiseResolve () {
if (executionAsyncId() === taskId) resolveCalls++;
}
});
hook.enable();
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
await pool.runTask('42');
hook.disable();
equal(initCalls, 1);
equal(beforeCalls, 1);
equal(afterCalls, 1);
equal(resolveCalls, 1);
});

81
my-app/node_modules/piscina/test/atomics-optimization.ts generated vendored Executable file
View file

@ -0,0 +1,81 @@
import Piscina from '..';
import { test } from 'tap';
import { resolve } from 'path';
test('coverage test for Atomics optimization', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/notify-then-sleep-or.ts'),
minThreads: 2,
maxThreads: 2,
concurrentTasksPerWorker: 2
});
const tasks = [];
let v : number;
// Post 4 tasks, and wait for all of them to be ready.
const i32array = new Int32Array(new SharedArrayBuffer(4));
for (let index = 0; index < 4; index++) {
tasks.push(pool.runTask({ i32array, index }));
}
// Wait for 2 tasks to enter 'wait' state.
do {
v = Atomics.load(i32array, 0);
if (popcount8(v) >= 2) break;
Atomics.wait(i32array, 0, v);
} while (true);
// The check above could also be !== 2 but it's hard to get things right
// sometimes and this gives us a nice assertion. Basically, at this point
// exactly 2 tasks should be in Atomics.wait() state.
equal(popcount8(v), 2);
// Wake both tasks up as simultaneously as possible. The other 2 tasks should
// then start executing.
Atomics.store(i32array, 0, 0);
Atomics.notify(i32array, 0, Infinity);
// Wait for the other 2 tasks to enter 'wait' state.
do {
v = Atomics.load(i32array, 0);
if (popcount8(v) >= 2) break;
Atomics.wait(i32array, 0, v);
} while (true);
// At this point, the first two tasks are definitely finished and have
// definitely posted results back to the main thread, and the main thread
// has definitely not received them yet, meaning that the Atomics check will
// be used. Making sure that that works is the point of this test.
// Wake up the remaining 2 tasks in order to make sure that the test finishes.
// Do the same consistency check beforehand as above.
equal(popcount8(v), 2);
Atomics.store(i32array, 0, 0);
Atomics.notify(i32array, 0, Infinity);
await Promise.all(tasks);
});
// Inefficient but straightforward 8-bit popcount
function popcount8 (v : number) : number {
v &= 0xff;
if (v & 0b11110000) return popcount8(v >>> 4) + popcount8(v & 0xb00001111);
if (v & 0b00001100) return popcount8(v >>> 2) + popcount8(v & 0xb00000011);
if (v & 0b00000010) return popcount8(v >>> 1) + popcount8(v & 0xb00000001);
return v;
}
test('avoids unbounded recursion', async () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/simple-isworkerthread.ts'),
minThreads: 2,
maxThreads: 2
});
const tasks = [];
for (let i = 1; i <= 10000; i++) {
tasks.push(pool.runTask(null));
}
await Promise.all(tasks);
});

17
my-app/node_modules/piscina/test/console-log.ts generated vendored Executable file
View file

@ -0,0 +1,17 @@
import concat from 'concat-stream';
import { spawn } from 'child_process';
import { resolve } from 'path';
import { test } from 'tap';
test('console.log() calls are not blocked by Atomics.wait()', async ({ equal }) => {
const proc = spawn(process.execPath, [
...process.execArgv, resolve(__dirname, 'fixtures/console-log.ts')
], {
stdio: ['inherit', 'pipe', 'inherit']
});
const data = await new Promise((resolve) => {
proc.stdout.setEncoding('utf8').pipe(concat(resolve));
});
equal(data, 'A\nB\n');
});

9
my-app/node_modules/piscina/test/fixtures/console-log.ts generated vendored Executable file
View file

@ -0,0 +1,9 @@
import Piscina from '../..';
import { resolve } from 'path';
const pool = new Piscina({
filename: resolve(__dirname, 'eval.js'),
maxThreads: 1
});
pool.runTask('console.log("A"); console.log("B");');

12
my-app/node_modules/piscina/test/fixtures/esm-async.mjs generated vendored Executable file
View file

@ -0,0 +1,12 @@
import util from 'util';
const sleep = util.promisify(setTimeout);
// eslint-disable-next-line no-eval
function handler (code) { return eval(code); }
async function load () {
await sleep(100);
return handler;
}
export default load();

2
my-app/node_modules/piscina/test/fixtures/esm-export.mjs generated vendored Executable file
View file

@ -0,0 +1,2 @@
// eslint-disable-next-line no-eval
export default function (code) { return eval(code); };

14
my-app/node_modules/piscina/test/fixtures/eval-async.js generated vendored Executable file
View file

@ -0,0 +1,14 @@
'use strict';
const { promisify } = require('util');
const sleep = promisify(setTimeout);
// eslint-disable-next-line no-eval
function handler (code) { return eval(code); }
async function load () {
await sleep(100);
return handler;
}
module.exports = load();

2
my-app/node_modules/piscina/test/fixtures/eval.js generated vendored Executable file
View file

@ -0,0 +1,2 @@
// eslint-disable-next-line no-eval
module.exports = function (code) { return eval(code); };

10
my-app/node_modules/piscina/test/fixtures/move.ts generated vendored Executable file
View file

@ -0,0 +1,10 @@
import Piscina from '../..';
import assert from 'assert';
import { types } from 'util';
export default function (moved) {
if (moved !== undefined) {
assert(types.isAnyArrayBuffer(moved));
}
return Piscina.move(new ArrayBuffer(10));
}

10
my-app/node_modules/piscina/test/fixtures/multiple.js generated vendored Executable file
View file

@ -0,0 +1,10 @@
'use strict';
function a () { return 'a'; }
function b () { return 'b'; }
a.a = a;
a.b = b;
module.exports = a;

View file

@ -0,0 +1,10 @@
// Set the index-th bith in i32array[0], then wait for it to be un-set again.
module.exports = function ({ i32array, index }) {
Atomics.or(i32array, 0, 1 << index);
Atomics.notify(i32array, 0, Infinity);
do {
const v = Atomics.load(i32array, 0);
if (!(v & (1 << index))) break;
Atomics.wait(i32array, 0, v);
} while (true);
};

View file

@ -0,0 +1,5 @@
module.exports = function (i32array) {
Atomics.store(i32array, 0, 1);
Atomics.notify(i32array, 0, Infinity);
Atomics.wait(i32array, 0, 1);
};

View file

@ -0,0 +1,8 @@
'use strict';
module.exports = () => {
const array = [];
while (true) {
array.push([array]);
}
};

View file

@ -0,0 +1,18 @@
'use strict';
const Piscina = require('../../dist/src');
let time;
module.exports = {
send: async () => {
const data = new ArrayBuffer(128);
try {
return Piscina.move(data);
} finally {
setTimeout(() => { time = data.byteLength; }, 1000);
}
},
get: () => {
return time;
}
};

View file

@ -0,0 +1,38 @@
'use strict';
const Piscina = require('../../dist/src');
class Shared {
constructor (data) {
this.name = 'shared';
this.data = data;
}
get [Piscina.transferableSymbol] () {
return [this.data];
}
get [Piscina.valueSymbol] () {
return { name: this.name, data: this.data };
}
make () {
return Piscina.move(this);
}
}
let time;
module.exports = {
send: async () => {
const data = new ArrayBuffer(128);
const shared = new Shared(data);
try {
return shared.make();
} finally {
setTimeout(() => { time = data.byteLength; }, 1000);
}
},
get: () => {
return time;
}
};

View file

@ -0,0 +1,6 @@
import Piscina from '../..';
import assert from 'assert';
assert.strictEqual(Piscina.isWorkerThread, true);
export default function () { return 'done'; }

View file

@ -0,0 +1,6 @@
import Piscina from '../..';
import assert from 'assert';
assert.strictEqual(Piscina.workerData, 'ABC');
export default function () { return 'done'; }

12
my-app/node_modules/piscina/test/fixtures/sleep.js generated vendored Executable file
View file

@ -0,0 +1,12 @@
'use strict';
const { promisify } = require('util');
const sleep = promisify(setTimeout);
const buf = new Uint32Array(new SharedArrayBuffer(4));
module.exports = async ({ time = 100, a }) => {
await sleep(time);
const ret = Atomics.exchange(buf, 0, a);
return ret;
};

View file

@ -0,0 +1,5 @@
module.exports = function (i32array) {
Atomics.wait(i32array, 0, 0);
Atomics.store(i32array, 0, -1);
Atomics.notify(i32array, 0, Infinity);
};

11
my-app/node_modules/piscina/test/fixtures/wait-for-others.ts generated vendored Executable file
View file

@ -0,0 +1,11 @@
import { threadId } from 'worker_threads';
module.exports = async function ([i32array, n]) {
Atomics.add(i32array, 0, 1);
Atomics.notify(i32array, 0, Infinity);
let lastSeenValue;
while ((lastSeenValue = Atomics.load(i32array, 0)) < n) {
Atomics.wait(i32array, 0, lastSeenValue);
}
return threadId;
};

31
my-app/node_modules/piscina/test/histogram.ts generated vendored Executable file
View file

@ -0,0 +1,31 @@
import Piscina from '..';
import { test } from 'tap';
import { resolve } from 'path';
test('pool will maintain run and wait time histograms', async ({ equal, ok }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
const tasks = [];
for (let n = 0; n < 10; n++) {
tasks.push(pool.runTask('42'));
}
await Promise.all(tasks);
const waitTime = pool.waitTime as any;
ok(waitTime);
equal(typeof waitTime.average, 'number');
equal(typeof waitTime.mean, 'number');
equal(typeof waitTime.stddev, 'number');
equal(typeof waitTime.min, 'number');
equal(typeof waitTime.max, 'number');
const runTime = pool.runTime as any;
ok(runTime);
equal(typeof runTime.average, 'number');
equal(typeof runTime.mean, 'number');
equal(typeof runTime.stddev, 'number');
equal(typeof runTime.min, 'number');
equal(typeof runTime.max, 'number');
});

44
my-app/node_modules/piscina/test/idle-timeout.ts generated vendored Executable file
View file

@ -0,0 +1,44 @@
import Piscina from '..';
import { test } from 'tap';
import { resolve } from 'path';
import { promisify } from 'util';
const delay = promisify(setTimeout);
test('idle timeout will let go of threads early', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/wait-for-others.ts'),
idleTimeout: 500,
minThreads: 1,
maxThreads: 2
});
equal(pool.threads.length, 1);
const buffer = new Int32Array(new SharedArrayBuffer(4));
const firstTasks = [
pool.runTask([buffer, 2]),
pool.runTask([buffer, 2])
];
equal(pool.threads.length, 2);
const earlyThreadIds = await Promise.all(firstTasks);
equal(pool.threads.length, 2);
await delay(2000);
equal(pool.threads.length, 1);
const secondTasks = [
pool.runTask([buffer, 4]),
pool.runTask([buffer, 4])
];
equal(pool.threads.length, 2);
const lateThreadIds = await Promise.all(secondTasks);
// One thread should have been idle in between and exited, one should have
// been reused.
equal(earlyThreadIds.length, 2);
equal(lateThreadIds.length, 2);
equal(new Set([...earlyThreadIds, ...lateThreadIds]).size, 3);
});

21
my-app/node_modules/piscina/test/load-with-esm.ts generated vendored Executable file
View file

@ -0,0 +1,21 @@
import { test } from 'tap';
const importESM : (specifier : string) => Promise<any> =
// eslint-disable-next-line no-eval
eval('(specifier) => import(specifier)');
test('Piscina is default export', {}, async ({ equal }) => {
equal((await importESM('piscina')).default, require('../'));
});
test('Exports match own property names', {}, async ({ strictSame }) => {
// Check that version, workerData, etc. are re-exported.
const exported = new Set(Object.getOwnPropertyNames(await importESM('piscina')));
const required = new Set(Object.getOwnPropertyNames(require('../')));
// Remove constructor properties + default export.
for (const k of ['prototype', 'length', 'name']) required.delete(k);
exported.delete('default');
strictSame(exported, required);
});

19
my-app/node_modules/piscina/test/messages.ts generated vendored Executable file
View file

@ -0,0 +1,19 @@
import Piscina from '../dist/src';
import { test } from 'tap';
import { resolve } from 'path';
import { once } from 'events';
test('Pool receive message from workers', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
const messagePromise = once(pool, 'message');
const taskResult = pool.runTask(`
require('worker_threads').parentPort.postMessage("some message");
42
`);
equal(await taskResult, 42);
equal((await messagePromise)[0], 'some message');
});

107
my-app/node_modules/piscina/test/move-test.ts generated vendored Executable file
View file

@ -0,0 +1,107 @@
import Piscina from '..';
import {
isMovable,
markMovable,
isTransferable
} from '../dist/src/common';
import { test } from 'tap';
import { types } from 'util';
import { MessageChannel, MessagePort } from 'worker_threads';
import { resolve } from 'path';
const {
transferableSymbol,
valueSymbol
} = Piscina;
test('Marking an object as movable works as expected', async ({ ok }) => {
const obj : any = {
get [transferableSymbol] () : object { return {}; },
get [valueSymbol] () : object { return {}; }
};
ok(isTransferable(obj));
ok(!isMovable(obj)); // It's not movable initially
markMovable(obj);
ok(isMovable(obj)); // It is movable now
});
test('Marking primitives and null works as expected', async ({ equal }) => {
equal(Piscina.move(null), null);
equal(Piscina.move(1 as any), 1);
equal(Piscina.move(false as any), false);
equal(Piscina.move('test' as any), 'test');
});
test('Using Piscina.move() returns a movable object', async ({ ok }) => {
const obj : any = {
get [transferableSymbol] () : object { return {}; },
get [valueSymbol] () : object { return {}; }
};
ok(!isMovable(obj)); // It's not movable initially
const movable = Piscina.move(obj);
ok(isMovable(movable)); // It is movable now
});
test('Using ArrayBuffer works as expected', async ({ ok, equal }) => {
const ab = new ArrayBuffer(5);
const movable = Piscina.move(ab);
ok(isMovable(movable));
ok(types.isAnyArrayBuffer(movable[valueSymbol]));
ok(types.isAnyArrayBuffer(movable[transferableSymbol]));
equal(movable[transferableSymbol], ab);
});
test('Using TypedArray works as expected', async ({ ok, equal }) => {
const ab = new Uint8Array(5);
const movable = Piscina.move(ab);
ok(isMovable(movable));
ok((types as any).isArrayBufferView(movable[valueSymbol]));
ok(types.isAnyArrayBuffer(movable[transferableSymbol]));
equal(movable[transferableSymbol], ab.buffer);
});
test('Using MessagePort works as expected', async ({ ok, equal }) => {
const mc = new MessageChannel();
const movable = Piscina.move(mc.port1);
ok(isMovable(movable));
ok(movable[valueSymbol] instanceof MessagePort);
ok(movable[transferableSymbol] instanceof MessagePort);
equal(movable[transferableSymbol], mc.port1);
});
test('Moving works', async ({ equal, ok }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/move.ts')
});
{
const ab = new ArrayBuffer(10);
const ret = await pool.runTask(Piscina.move(ab));
equal(ab.byteLength, 0); // It was moved
ok(types.isAnyArrayBuffer(ret));
}
{
// Test with empty transferList
const ab = new ArrayBuffer(10);
const ret = await pool.runTask(Piscina.move(ab), []);
equal(ab.byteLength, 0); // It was moved
ok(types.isAnyArrayBuffer(ret));
}
{
// Test with empty transferList
const ab = new ArrayBuffer(10);
const ret = await pool.run(Piscina.move(ab));
equal(ab.byteLength, 0); // It was moved
ok(types.isAnyArrayBuffer(ret));
}
{
// Test with empty transferList
const ab = new ArrayBuffer(10);
const ret = await pool.run(Piscina.move(ab), { transferList: [] });
equal(ab.byteLength, 0); // It was moved
ok(types.isAnyArrayBuffer(ret));
}
});

31
my-app/node_modules/piscina/test/nice.ts generated vendored Executable file
View file

@ -0,0 +1,31 @@
import Piscina from '..';
import { resolve } from 'path';
import { test } from 'tap';
test('can set niceness for threads on Linux', {
skip: process.platform !== 'linux'
}, async ({ equal }) => {
const worker = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
niceIncrement: 5
});
// ts-ignore because the dependency is not installed on Windows.
// @ts-ignore
const currentNiceness = (await import('nice-napi')).default(0);
const result = await worker.runTask('require("nice-napi")()');
// niceness is capped to 19 on Linux.
const expected = Math.min(currentNiceness + 5, 19);
equal(result, expected);
});
test('setting niceness never does anything bad', async ({ equal }) => {
const worker = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
niceIncrement: 5
});
const result = await worker.runTask('42');
equal(result, 42);
});

129
my-app/node_modules/piscina/test/option-validation.ts generated vendored Executable file
View file

@ -0,0 +1,129 @@
import Piscina from '..';
import { test } from 'tap';
test('filename cannot be non-null/non-string', async ({ throws }) => {
throws(() => new Piscina(({
filename: 12
}) as any), /options.filename must be a string or null/);
});
test('name cannot be non-null/non-string', async ({ throws }) => {
throws(() => new Piscina(({
name: 12
}) as any), /options.name must be a string or null/);
});
test('minThreads must be non-negative integer', async ({ throws }) => {
throws(() => new Piscina(({
minThreads: -1
}) as any), /options.minThreads must be a non-negative integer/);
throws(() => new Piscina(({
minThreads: 'string'
}) as any), /options.minThreads must be a non-negative integer/);
});
test('maxThreads must be positive integer', async ({ throws }) => {
throws(() => new Piscina(({
maxThreads: -1
}) as any), /options.maxThreads must be a positive integer/);
throws(() => new Piscina(({
maxThreads: 0
}) as any), /options.maxThreads must be a positive integer/);
throws(() => new Piscina(({
maxThreads: 'string'
}) as any), /options.maxThreads must be a positive integer/);
});
test('concurrentTasksPerWorker must be positive integer', async ({ throws }) => {
throws(() => new Piscina(({
concurrentTasksPerWorker: -1
}) as any), /options.concurrentTasksPerWorker must be a positive integer/);
throws(() => new Piscina(({
concurrentTasksPerWorker: 0
}) as any), /options.concurrentTasksPerWorker must be a positive integer/);
throws(() => new Piscina(({
concurrentTasksPerWorker: 'string'
}) as any), /options.concurrentTasksPerWorker must be a positive integer/);
});
test('idleTimeout must be non-negative integer', async ({ throws }) => {
throws(() => new Piscina(({
idleTimeout: -1
}) as any), /options.idleTimeout must be a non-negative integer/);
throws(() => new Piscina(({
idleTimeout: 'string'
}) as any), /options.idleTimeout must be a non-negative integer/);
});
test('maxQueue must be non-negative integer', async ({ throws, equal }) => {
throws(() => new Piscina(({
maxQueue: -1
}) as any), /options.maxQueue must be a non-negative integer/);
throws(() => new Piscina(({
maxQueue: 'string'
}) as any), /options.maxQueue must be a non-negative integer/);
const p = new Piscina({ maxQueue: 'auto', maxThreads: 2 });
equal(p.options.maxQueue, 4);
});
test('useAtomics must be a boolean', async ({ throws }) => {
throws(() => new Piscina(({
useAtomics: -1
}) as any), /options.useAtomics must be a boolean/);
throws(() => new Piscina(({
useAtomics: 'string'
}) as any), /options.useAtomics must be a boolean/);
});
test('resourceLimits must be an object', async ({ throws }) => {
throws(() => new Piscina(({
resourceLimits: 0
}) as any), /options.resourceLimits must be an object/);
});
test('taskQueue must be a TaskQueue object', async ({ throws }) => {
throws(() => new Piscina(({
taskQueue: 0
}) as any), /options.taskQueue must be a TaskQueue object/);
throws(() => new Piscina(({
taskQueue: 'test'
}) as any), /options.taskQueue must be a TaskQueue object/);
throws(() => new Piscina(({
taskQueue: null
}) as any), /options.taskQueue must be a TaskQueue object/);
throws(() => new Piscina(({
taskQueue: new Date()
}) as any), /options.taskQueue must be a TaskQueue object/);
throws(() => new Piscina(({
taskQueue: { } as any
}) as any), /options.taskQueue must be a TaskQueue object/);
});
test('niceIncrement must be non-negative integer', async ({ throws }) => {
throws(() => new Piscina(({
niceIncrement: -1
}) as any), /options.niceIncrement must be a non-negative integer/);
throws(() => new Piscina(({
niceIncrement: 'string'
}) as any), /options.niceIncrement must be a non-negative integer/);
});
test('trackUnmanagedFds must be a boolean', async ({ throws }) => {
throws(() => new Piscina(({
trackUnmanagedFds: -1
}) as any), /options.trackUnmanagedFds must be a boolean/);
throws(() => new Piscina(({
trackUnmanagedFds: 'string'
}) as any), /options.trackUnmanagedFds must be a boolean/);
});

99
my-app/node_modules/piscina/test/pool-close.ts generated vendored Executable file
View file

@ -0,0 +1,99 @@
import { test } from 'tap';
import Piscina from '..';
import { resolve } from 'path';
import { once } from 'events';
test('close()', async (t) => {
t.test('no pending tasks', async (t) => {
const pool = new Piscina({ filename: resolve(__dirname, 'fixtures/sleep.js') });
await pool.close();
t.pass('pool closed successfully');
});
t.test('queued tasks waits for all tasks to complete', async (t) => {
const pool = new Piscina({ filename: resolve(__dirname, 'fixtures/sleep.js'), maxThreads: 1 });
const task1 = pool.run({ time: 100 });
const task2 = pool.run({ time: 100 });
setImmediate(() => t.resolves(pool.close(), 'close is resolved when all running tasks are completed'));
await Promise.all([
t.resolves(once(pool, 'close'), 'handler is called when pool is closed'),
t.resolves(task1, 'complete running task'),
t.resolves(task2, 'complete running task')
]);
});
t.test('abort any task enqueued during closing up', async (t) => {
const pool = new Piscina({ filename: resolve(__dirname, 'fixtures/sleep.js'), maxThreads: 1 });
setImmediate(() => {
t.resolves(pool.close(), 'close is resolved when running tasks are completed');
t.resolves(pool.run({ time: 1000 }).then(null, err => {
t.equal(err.message, 'The task has been aborted');
t.equal(err.cause, 'queue is closing up');
}));
});
await t.resolves(pool.run({ time: 100 }), 'complete running task');
});
});
test('close({force: true})', async (t) => {
t.test('queued tasks waits for all tasks already running and aborts tasks that are not started yet', async (t) => {
const pool = new Piscina({ filename: resolve(__dirname, 'fixtures/sleep.js'), maxThreads: 1, concurrentTasksPerWorker: 1 });
const task1 = pool.run({ time: 1000 });
const task2 = pool.run({ time: 200 });
setImmediate(() => t.resolves(pool.close({ force: true }), 'close is resolved when all running tasks are completed'));
await Promise.all([
t.resolves(once(pool, 'close'), 'handler is called when pool is closed'),
t.resolves(task1, 'complete running task'),
t.resolves(task2.then(null, err => {
t.equal(err.message, 'The task has been aborted');
t.equal(err.cause, 'pool is closed');
}))
]);
});
t.test('queued tasks waits for all tasks already running and aborts tasks that are not started yet', async (t) => {
const pool = new Piscina({ filename: resolve(__dirname, 'fixtures/sleep.js'), maxThreads: 1, concurrentTasksPerWorker: 2 });
const task1 = pool.run({ time: 500 });
const task2 = pool.run({ time: 100 });
const task3 = pool.run({ time: 100 });
const task4 = pool.run({ time: 100 });
setImmediate(() => t.resolves(pool.close({ force: true }), 'close is resolved when all running tasks are completed'));
await Promise.all([
t.resolves(once(pool, 'close'), 'handler is called when pool is closed'),
t.resolves(task1, 'complete running task'),
t.resolves(task2, 'complete running task'),
t.rejects(task3, /The task has been aborted/, 'abort task that are not started yet'),
t.rejects(task4, /The task has been aborted/, 'abort task that are not started yet')
]);
});
});
test('timed out close operation destroys the pool', async (t) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/sleep.js'),
maxThreads: 1,
closeTimeout: 500
});
const task1 = pool.run({ time: 5000 });
const task2 = pool.run({ time: 5000 });
setImmediate(() => t.resolves(pool.close(), 'close is resolved on timeout'));
await Promise.all([
t.resolves(once(pool, 'error'), 'error handler is called on timeout'),
t.rejects(task1, /Terminating worker thread/, 'task is aborted due to timeout'),
t.rejects(task2, /Terminating worker thread/, 'task is aborted due to timeout')
]);
});

11
my-app/node_modules/piscina/test/pool-destroy.ts generated vendored Executable file
View file

@ -0,0 +1,11 @@
import Piscina from '..';
import { test } from 'tap';
import { resolve } from 'path';
test('can destroy pool while tasks are running', async ({ rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
setImmediate(() => pool.destroy());
await rejects(pool.runTask('while(1){}'), /Terminating worker thread/);
});

207
my-app/node_modules/piscina/test/post-task.ts generated vendored Executable file
View file

@ -0,0 +1,207 @@
import { MessageChannel } from 'worker_threads';
import { cpus } from 'os';
import Piscina from '..';
import { test } from 'tap';
import { resolve } from 'path';
test('postTask() can transfer ArrayBuffer instances', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/simple-isworkerthread.ts')
});
const ab = new ArrayBuffer(40);
await pool.runTask({ ab }, [ab]);
equal(pool.completed, 1);
equal(ab.byteLength, 0);
});
test('postTask() can transfer ArrayBuffer instances', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/simple-isworkerthread.ts')
});
const ab = new ArrayBuffer(40);
await pool.run({ ab }, { transferList: [ab] });
equal(pool.completed, 1);
equal(ab.byteLength, 0);
});
test('postTask() cannot clone build-in objects', async ({ rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/simple-isworkerthread.ts')
});
const obj = new MessageChannel().port1;
rejects(pool.runTask({ obj }));
});
test('postTask() resolves with a rejection when the handler rejects', async ({ rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
rejects(pool.runTask('Promise.reject(new Error("foo"))'), /foo/);
});
test('postTask() resolves with a rejection when the handler throws', async ({ rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
rejects(pool.runTask('throw new Error("foo")'), /foo/);
});
test('postTask() validates transferList', async ({ rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
rejects(pool.runTask('0', 42 as any),
/transferList argument must be an Array/);
rejects(pool.run('0', { transferList: 42 as any }),
/transferList argument must be an Array/);
});
test('postTask() validates filename', async ({ rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
rejects(pool.runTask('0', [], 42 as any),
/filename argument must be a string/);
rejects(pool.run('0', { filename: 42 as any }),
/filename argument must be a string/);
});
test('postTask() validates name', async ({ rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
rejects(pool.run('0', { name: 42 as any }),
/name argument must be a string/);
});
test('postTask() validates abortSignal', async ({ rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
rejects(pool.runTask('0', [], undefined, 42 as any),
/signal argument must be an object/);
rejects(pool.run('0', { signal: 42 as any }),
/signal argument must be an object/);
});
test('Piscina emits drain', async ({ ok, notOk }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
let drained = false;
let needsDrain = true;
pool.on('drain', () => {
drained = true;
needsDrain = pool.needsDrain;
});
await Promise.all([pool.run('123'), pool.run('123')]);
ok(drained);
notOk(needsDrain);
});
test('Piscina exposes/emits needsDrain to true when capacity is exceeded', async ({ ok }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
maxQueue: 3,
maxThreads: 1
});
let triggered = false;
let drained = false;
pool.once('drain', () => {
drained = true;
});
pool.once('needsDrain', () => {
triggered = true;
});
pool.run('123');
pool.run('123');
pool.run('123');
pool.run('123');
ok(pool.needsDrain);
ok(triggered);
ok(drained);
});
test('Piscina can use async loaded workers', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval-async.js')
});
equal(await pool.runTask('1'), 1);
});
test('Piscina can use async loaded esm workers', {}, async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/esm-async.mjs')
});
equal(await pool.runTask('1'), 1);
});
test('Piscina.run options is correct type', async ({ rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
rejects(pool.run(42, 1 as any), /options must be an object/);
});
test('Piscina.maxThreads should return the max number of threads to be used (default)', ({ equal, plan }) => {
plan(1);
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
const maxThreads = (cpus().length || 1) * 1.5;
equal(pool.maxThreads, maxThreads);
});
test('Piscina.minThreads should return the max number of threads to be used (custom)', ({ equal, plan }) => {
const maxThreads = 3;
const pool = new Piscina({
maxThreads,
filename: resolve(__dirname, 'fixtures/eval.js')
});
plan(1);
equal(pool.maxThreads, maxThreads);
});
test('Piscina.minThreads should return the max number of threads to be used (default)', ({ equal, plan }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
const minThreads = Math.max((cpus().length || 1) / 2, 1);
plan(1);
equal(pool.minThreads, minThreads);
});
test('Piscina.minThreads should return the max number of threads to be used (custom)', ({ equal, plan }) => {
const minThreads = 2;
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
minThreads
});
plan(1);
equal(pool.minThreads, minThreads);
});

199
my-app/node_modules/piscina/test/simple-test.ts generated vendored Executable file
View file

@ -0,0 +1,199 @@
import Piscina from '..';
import { test } from 'tap';
import { version } from '../package.json';
import { pathToFileURL } from 'url';
import { resolve } from 'path';
import { EventEmitter } from 'events';
test('Piscina is exposed on export', async ({ equal }) => {
equal(Piscina.version, version);
});
test('Piscina is exposed on itself', async ({ equal }) => {
equal(Piscina.Piscina, Piscina);
});
test('Piscina.isWorkerThread has the correct value', async ({ equal }) => {
equal(Piscina.isWorkerThread, false);
});
test('Piscina.isWorkerThread has the correct value (worker)', async ({ equal }) => {
const worker = new Piscina({
filename: resolve(__dirname, 'fixtures/simple-isworkerthread.ts')
});
const result = await worker.runTask(null);
equal(result, 'done');
});
test('Piscina instance is an EventEmitter', async ({ ok }) => {
const piscina = new Piscina();
ok(piscina instanceof EventEmitter);
});
test('Piscina constructor options are correctly set', async ({ equal }) => {
const piscina = new Piscina({
minThreads: 10,
maxThreads: 20,
maxQueue: 30
});
equal(piscina.options.minThreads, 10);
equal(piscina.options.maxThreads, 20);
equal(piscina.options.maxQueue, 30);
});
test('trivial eval() handler works', async ({ equal }) => {
const worker = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
const result = await worker.runTask('42');
equal(result, 42);
});
test('async eval() handler works', async ({ equal }) => {
const worker = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
const result = await worker.runTask('Promise.resolve(42)');
equal(result, 42);
});
test('filename can be provided while posting', async ({ equal }) => {
const worker = new Piscina();
const result = await worker.runTask(
'Promise.resolve(42)',
resolve(__dirname, 'fixtures/eval.js'));
equal(result, 42);
});
test('filename can be null when initially provided', async ({ equal }) => {
const worker = new Piscina({ filename: null });
const result = await worker.runTask(
'Promise.resolve(42)',
resolve(__dirname, 'fixtures/eval.js'));
equal(result, 42);
});
test('filename must be provided while posting', async ({ rejects }) => {
const worker = new Piscina();
rejects(worker.runTask('doesnt matter'),
/filename must be provided to run\(\) or in options object/);
});
test('passing env to workers works', async ({ same }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
env: { A: 'foo' }
});
const env = await pool.runTask('({...process.env})');
same(env, { A: 'foo' });
});
test('passing argv to workers works', async ({ same }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
argv: ['a', 'b', 'c']
});
const env = await pool.runTask('process.argv.slice(2)');
same(env, ['a', 'b', 'c']);
});
test('passing execArgv to workers works', async ({ same }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
execArgv: ['--no-warnings']
});
const env = await pool.runTask('process.execArgv');
same(env, ['--no-warnings']);
});
test('passing valid workerData works', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/simple-workerdata.ts'),
workerData: 'ABC'
});
equal(Piscina.workerData, undefined);
await pool.runTask(null);
});
test('passing invalid workerData does not work', async ({ throws }) => {
throws(() => new Piscina(({
filename: resolve(__dirname, 'fixtures/simple-workerdata.ts'),
workerData: {
hello () {}
}
}) as any), /could not be cloned./);
});
test('filename can be a file:// URL', async ({ equal }) => {
const worker = new Piscina({
filename: pathToFileURL(resolve(__dirname, 'fixtures/eval.js')).href
});
const result = await worker.runTask('42');
equal(result, 42);
});
test('filename can be a file:// URL to an ESM module', {}, async ({ equal }) => {
const worker = new Piscina({
filename: pathToFileURL(resolve(__dirname, 'fixtures/esm-export.mjs')).href
});
const result = await worker.runTask('42');
equal(result, 42);
});
test('duration and utilization calculations work', async ({ equal, ok }) => {
const worker = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
// Initial utilization is always 0
equal(worker.utilization, 0);
await Promise.all([
worker.runTask('42'),
worker.runTask('41'),
worker.runTask('40')
]);
// utilization is going to be some non-deterministic value
// between 0 and 1. It should not be zero at this point
// because tasks have run, but it should also never be 1
ok(worker.utilization > 0);
ok(worker.utilization < 1);
// Duration must be non-zero.
ok(worker.duration > 0);
});
test('run works also', async () => {
const worker = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
await worker.run(42);
});
test('named tasks work', async ({ equal }) => {
const worker = new Piscina({
filename: resolve(__dirname, 'fixtures/multiple.js')
});
equal(await worker.run({}, { name: 'a' }), 'a');
equal(await worker.run({}, { name: 'b' }), 'b');
equal(await worker.run({}), 'a');
});
test('named tasks work', async ({ equal }) => {
const worker = new Piscina({
filename: resolve(__dirname, 'fixtures/multiple.js'),
name: 'b'
});
equal(await worker.run({}, { name: 'a' }), 'a');
equal(await worker.run({}, { name: 'b' }), 'b');
equal(await worker.run({}), 'b');
});

280
my-app/node_modules/piscina/test/task-queue.ts generated vendored Executable file
View file

@ -0,0 +1,280 @@
import Piscina from '..';
import { test } from 'tap';
import { resolve } from 'path';
import { Task, TaskQueue } from '../dist/src/common';
test('will put items into a task queue until they can run', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/wait-for-notify.ts'),
minThreads: 2,
maxThreads: 3
});
equal(pool.threads.length, 2);
equal(pool.queueSize, 0);
const buffers = [
new Int32Array(new SharedArrayBuffer(4)),
new Int32Array(new SharedArrayBuffer(4)),
new Int32Array(new SharedArrayBuffer(4)),
new Int32Array(new SharedArrayBuffer(4))
];
const results = [];
results.push(pool.runTask(buffers[0]));
equal(pool.threads.length, 2);
equal(pool.queueSize, 0);
results.push(pool.runTask(buffers[1]));
equal(pool.threads.length, 2);
equal(pool.queueSize, 0);
results.push(pool.runTask(buffers[2]));
equal(pool.threads.length, 3);
equal(pool.queueSize, 0);
results.push(pool.runTask(buffers[3]));
equal(pool.threads.length, 3);
equal(pool.queueSize, 1);
for (const buffer of buffers) {
Atomics.store(buffer, 0, 1);
Atomics.notify(buffer, 0, 1);
}
await results[0];
equal(pool.queueSize, 0);
await Promise.all(results);
});
test('will reject items over task queue limit', async ({ equal, rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
minThreads: 0,
maxThreads: 1,
maxQueue: 2
});
equal(pool.threads.length, 0);
equal(pool.queueSize, 0);
rejects(pool.runTask('while (true) {}'), /Terminating worker thread/);
equal(pool.threads.length, 1);
equal(pool.queueSize, 0);
rejects(pool.runTask('while (true) {}'), /Terminating worker thread/);
equal(pool.threads.length, 1);
equal(pool.queueSize, 1);
rejects(pool.runTask('while (true) {}'), /Terminating worker thread/);
equal(pool.threads.length, 1);
equal(pool.queueSize, 2);
rejects(pool.runTask('while (true) {}'), /Task queue is at limit/);
await pool.destroy();
});
test('will reject items when task queue is unavailable', async ({ equal, rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
minThreads: 0,
maxThreads: 1,
maxQueue: 0
});
equal(pool.threads.length, 0);
equal(pool.queueSize, 0);
rejects(pool.runTask('while (true) {}'), /Terminating worker thread/);
equal(pool.threads.length, 1);
equal(pool.queueSize, 0);
rejects(pool.runTask('while (true) {}'), /No task queue available and all Workers are busy/);
await pool.destroy();
});
test('will reject items when task queue is unavailable (fixed thread count)', async ({ equal, rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
minThreads: 1,
maxThreads: 1,
maxQueue: 0
});
equal(pool.threads.length, 1);
equal(pool.queueSize, 0);
rejects(pool.runTask('while (true) {}'), /Terminating worker thread/);
equal(pool.threads.length, 1);
equal(pool.queueSize, 0);
rejects(pool.runTask('while (true) {}'), /No task queue available and all Workers are busy/);
await pool.destroy();
});
test('tasks can share a Worker if requested (both tests blocking)', async ({ equal, rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/wait-for-notify.ts'),
minThreads: 0,
maxThreads: 1,
maxQueue: 0,
concurrentTasksPerWorker: 2
});
equal(pool.threads.length, 0);
equal(pool.queueSize, 0);
rejects(pool.runTask(new Int32Array(new SharedArrayBuffer(4))));
equal(pool.threads.length, 1);
equal(pool.queueSize, 0);
rejects(pool.runTask(new Int32Array(new SharedArrayBuffer(4))));
equal(pool.threads.length, 1);
equal(pool.queueSize, 0);
await pool.destroy();
});
test('tasks can share a Worker if requested (one test finishes)', async ({ equal, rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/wait-for-notify.ts'),
minThreads: 0,
maxThreads: 1,
maxQueue: 0,
concurrentTasksPerWorker: 2
});
const buffers = [
new Int32Array(new SharedArrayBuffer(4)),
new Int32Array(new SharedArrayBuffer(4))
];
equal(pool.threads.length, 0);
equal(pool.queueSize, 0);
const firstTask = pool.runTask(buffers[0]);
equal(pool.threads.length, 1);
equal(pool.queueSize, 0);
rejects(pool.runTask(
'new Promise((resolve) => setTimeout(resolve, 1000000))',
resolve(__dirname, 'fixtures/eval.js')), /Terminating worker thread/);
equal(pool.threads.length, 1);
equal(pool.queueSize, 0);
Atomics.store(buffers[0], 0, 1);
Atomics.notify(buffers[0], 0, 1);
await firstTask;
equal(pool.threads.length, 1);
equal(pool.queueSize, 0);
await pool.destroy();
});
test('tasks can share a Worker if requested (both tests finish)', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/wait-for-notify.ts'),
minThreads: 1,
maxThreads: 1,
maxQueue: 0,
concurrentTasksPerWorker: 2
});
const buffers = [
new Int32Array(new SharedArrayBuffer(4)),
new Int32Array(new SharedArrayBuffer(4))
];
equal(pool.threads.length, 1);
equal(pool.queueSize, 0);
const firstTask = pool.runTask(buffers[0]);
equal(pool.threads.length, 1);
equal(pool.queueSize, 0);
const secondTask = pool.runTask(buffers[1]);
equal(pool.threads.length, 1);
equal(pool.queueSize, 0);
Atomics.store(buffers[0], 0, 1);
Atomics.store(buffers[1], 0, 1);
Atomics.notify(buffers[0], 0, 1);
Atomics.notify(buffers[1], 0, 1);
Atomics.wait(buffers[0], 0, 1);
Atomics.wait(buffers[1], 0, 1);
await firstTask;
equal(buffers[0][0], -1);
await secondTask;
equal(buffers[1][0], -1);
equal(pool.threads.length, 1);
equal(pool.queueSize, 0);
});
test('custom task queue works', async ({ equal, ok }) => {
let sizeCalled : boolean = false;
let shiftCalled : boolean = false;
let pushCalled : boolean = false;
class CustomTaskPool implements TaskQueue {
tasks: Task[] = [];
get size () : number {
sizeCalled = true;
return this.tasks.length;
}
shift () : Task | null {
shiftCalled = true;
return this.tasks.length > 0 ? this.tasks.shift() as Task : null;
}
push (task : Task) : void {
pushCalled = true;
this.tasks.push(task);
ok(Piscina.queueOptionsSymbol in task);
if ((task as any).task.a === 3) {
equal(task[Piscina.queueOptionsSymbol], null);
} else {
equal(task[Piscina.queueOptionsSymbol].option,
(task as any).task.a);
}
}
remove (task : Task) : void {
const index = this.tasks.indexOf(task);
this.tasks.splice(index, 1);
}
};
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
taskQueue: new CustomTaskPool(),
// Setting maxThreads low enough to ensure we queue
maxThreads: 1,
minThreads: 1
});
function makeTask (task, option) {
return { ...task, [Piscina.queueOptionsSymbol]: { option } };
}
const ret = await Promise.all([
pool.runTask(makeTask({ a: 1 }, 1)),
pool.runTask(makeTask({ a: 2 }, 2)),
pool.runTask({ a: 3 }) // No queueOptionsSymbol attached
]);
equal(ret[0].a, 1);
equal(ret[1].a, 2);
equal(ret[2].a, 3);
ok(sizeCalled);
ok(pushCalled);
ok(shiftCalled);
});

View file

@ -0,0 +1,29 @@
import Piscina from '../dist/src';
import { test } from 'tap';
import { resolve } from 'path';
function wait () {
return new Promise((resolve) => setTimeout(resolve, 1500));
}
test('transferable objects must be transferred', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/send-buffer-then-get-length.js'),
useAtomics: false
});
await pool.run({}, { name: 'send' });
await wait();
const after = await pool.run({}, { name: 'get' });
equal(after, 0);
});
test('objects that implement transferable must be transferred', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/send-transferrable-then-get-length.js'),
useAtomics: false
});
await pool.run({}, { name: 'send' });
await wait();
const after = await pool.run({}, { name: 'get' });
equal(after, 0);
});

34
my-app/node_modules/piscina/test/test-resourcelimits.ts generated vendored Executable file
View file

@ -0,0 +1,34 @@
import Piscina from '..';
import { test } from 'tap';
import { resolve } from 'path';
test('resourceLimits causes task to reject', async ({ equal, rejects }) => {
const worker = new Piscina({
filename: resolve(__dirname, 'fixtures/resource-limits.js'),
resourceLimits: {
maxOldGenerationSizeMb: 16,
maxYoungGenerationSizeMb: 4,
codeRangeSizeMb: 16
}
});
worker.on('error', () => {
// Ignore any additional errors that may occur.
// This may happen because when the Worker is
// killed a new worker is created that may hit
// the memory limits immediately. When that
// happens, there is no associated Promise to
// reject so we emit an error event instead.
// We don't care so much about that here. We
// could potentially avoid the issue by setting
// higher limits above but rather than try to
// guess at limits that may work consistently,
// let's just ignore the additional error for
// now.
});
const limits : any = worker.options.resourceLimits;
equal(limits.maxOldGenerationSizeMb, 16);
equal(limits.maxYoungGenerationSizeMb, 4);
equal(limits.codeRangeSizeMb, 16);
rejects(worker.runTask(null),
/Worker terminated due to reaching memory limit: JS heap out of memory/);
});

View file

@ -0,0 +1,86 @@
import Piscina from '..';
import { test } from 'tap';
import { resolve } from 'path';
import { once } from 'events';
test('uncaught exception resets Worker', async ({ rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
await rejects(pool.runTask('throw new Error("not_caught")'), /not_caught/);
});
test('uncaught exception in immediate resets Worker', async ({ rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
await rejects(
pool.runTask(`
setImmediate(() => { throw new Error("not_caught") });
new Promise(() => {}) /* act as if we were doing some work */
`), /not_caught/);
});
test('uncaught exception in immediate after task yields error event', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
maxThreads: 1,
useAtomics: false
});
const errorEvent : Promise<Error[]> = once(pool, 'error');
const taskResult = pool.runTask(`
setTimeout(() => { throw new Error("not_caught") }, 500);
42
`);
equal(await taskResult, 42);
// Hack a bit to make sure we get the 'exit'/'error' events.
equal(pool.threads.length, 1);
pool.threads[0].ref();
// This is the main assertion here.
equal((await errorEvent)[0].message, 'not_caught');
});
test('exiting process resets worker', async ({ not, rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
minThreads: 1
});
const originalThreadId = pool.threads[0].threadId;
await rejects(pool.runTask('process.exit(1);'), /worker exited with code: 1/);
const newThreadId = pool.threads[0].threadId;
not(originalThreadId, newThreadId);
});
test('exiting process in immediate after task errors next task and resets worker', async ({ equal, not, rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval-async.js'),
minThreads: 1
});
const originalThreadId = pool.threads[0].threadId;
const taskResult = await pool.runTask(`
setTimeout(() => { process.exit(1); }, 50);
42
`);
equal(taskResult, 42);
await rejects(pool.runTask(`
'use strict';
const { promisify } = require('util');
const sleep = promisify(setTimeout);
async function _() {
await sleep(1000);
return 42
}
_();
`), /worker exited with code: 1/);
const secondThreadId = pool.threads[0].threadId;
not(originalThreadId, secondThreadId);
});

64
my-app/node_modules/piscina/test/thread-count.ts generated vendored Executable file
View file

@ -0,0 +1,64 @@
import Piscina from '..';
import { cpus } from 'os';
import { test } from 'tap';
import { resolve } from 'path';
test('will start with minThreads and max out at maxThreads', async ({ equal, rejects }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
minThreads: 2,
maxThreads: 4
});
equal(pool.threads.length, 2);
rejects(pool.runTask('while(true) {}'));
equal(pool.threads.length, 2);
rejects(pool.runTask('while(true) {}'));
equal(pool.threads.length, 2);
rejects(pool.runTask('while(true) {}'));
equal(pool.threads.length, 3);
rejects(pool.runTask('while(true) {}'));
equal(pool.threads.length, 4);
rejects(pool.runTask('while(true) {}'));
equal(pool.threads.length, 4);
await pool.destroy();
});
test('low maxThreads sets minThreads', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
maxThreads: 1
});
equal(pool.threads.length, 1);
equal(pool.options.minThreads, 1);
equal(pool.options.maxThreads, 1);
});
test('high minThreads sets maxThreads', {
skip: cpus().length > 8
}, async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
minThreads: 16
});
equal(pool.threads.length, 16);
equal(pool.options.minThreads, 16);
equal(pool.options.maxThreads, 16);
});
test('conflicting min/max threads is error', async ({ throws }) => {
throws(() => new Piscina({
minThreads: 16,
maxThreads: 8
}), /options.minThreads and options.maxThreads must not conflict/);
});
test('thread count should be 0 upon destruction', async ({ equal }) => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
minThreads: 2,
maxThreads: 4
});
equal(pool.threads.length, 2);
await pool.destroy();
equal(pool.threads.length, 0);
});

10
my-app/node_modules/piscina/test/tsconfig.json generated vendored Executable file
View file

@ -0,0 +1,10 @@
{
"extends": "../tsconfig",
"compilerOptions": {
"strict": false,
"noImplicitAny": false
},
"include": [
"./**/*.ts"
]
}

24
my-app/node_modules/piscina/tsconfig.json generated vendored Executable file
View file

@ -0,0 +1,24 @@
{
"compilerOptions": {
"target": "es2019",
"module": "commonjs",
"moduleResolution": "node",
"lib": ["es2019"],
"outDir": "dist",
"rootDir": ".",
"declaration": true,
"sourceMap": true,
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": true,
"esModuleInterop": true,
"resolveJsonModule": true, /* Include modules imported with '.json' extension */
"forceConsistentCasingInFileNames": true /* Disallow inconsistently-cased references to the same file. */
},
"include": ["src"],
}