Nestia Guide Documents

Outline

Benchmark your backend server with e2e test functions.

If you’ve developed e2e test functions utilizing SDK library of @nestia/sdk, you can re-use those e2e test functions in the benchmark program supported by @nestia/benchmark. The benchmark program will run these e2e test functions in parellel and randomly to measure the performance of your backend server.

If you want to pre-exprience the benchmark program utliizng the e2e test functions of @nestia/sdk, visit below playground website. Also, here is the benchmark report example generated by the benchmark program of @nestia/benchmark executed in the below playgroud link.

💻 https://stackblitz.com/~/github.com/samchon/nestia-start

? Number of requests to make 1024
? Number of threads to use 4
? Number of simultaneous requests to make 32
████████████████████████████████████████ 100% | ETA: 0s | 3654/1024

Main Program

test/benchmark/index.ts

import { DynamicBenchmarker } from "@nestia/benchmark";
import cliProgress from "cli-progress";
import fs from "fs";
import os from "os";
import { IPointer } from "tstl";
 
import { MyBackend } from "../../src/MyBackend";
import { MyConfiguration } from "../../src/MyConfiguration";
import { MyGlobal } from "../../src/MyGlobal";
import { ArgumentParser } from "../helpers/ArgumentParser";
 
interface IOptions {
  include?: string[];
  exclude?: string[];
  count: number;
  threads: number;
  simultaneous: number;
}
 
const getOptions = () =>
  ArgumentParser.parse<IOptions>(async (command, prompt, action) => {
    // command.option("--mode <string>", "target mode");
    // command.option("--reset <true|false>", "reset local DB or not");
    command.option("--include <string...>", "include feature files");
    command.option("--exclude <string...>", "exclude feature files");
    command.option("--count <number>", "number of requests to make");
    command.option("--threads <number>", "number of threads to use");
    command.option(
      "--simultaneous <number>",
      "number of simultaneous requests to make",
    );
    return action(async (options) => {
      // if (typeof options.reset === "string")
      //     options.reset = options.reset === "true";
      // options.mode ??= await prompt.select("mode")("Select mode")([
      //     "LOCAL",
      //     "DEV",
      //     "REAL",
      // ]);
      // options.reset ??= await prompt.boolean("reset")("Reset local DB");
      options.count = Number(
        options.count ??
          (await prompt.number("count")("Number of requests to make")),
      );
      options.threads = Number(
        options.threads ??
          (await prompt.number("threads")("Number of threads to use")),
      );
      options.simultaneous = Number(
        options.simultaneous ??
          (await prompt.number("simultaneous")(
            "Number of simultaneous requests to make",
          )),
      );
      return options as IOptions;
    });
  });
 
const main = async (): Promise<void> => {
  // CONFIGURATIONS
  const options: IOptions = await getOptions();
  MyGlobal.testing = true;
 
  // BACKEND SERVER
  const backend: MyBackend = new MyBackend();
  await backend.open();
 
  // DO BENCHMARK
  const prev: IPointer<number> = { value: 0 };
  const bar: cliProgress.SingleBar = new cliProgress.SingleBar(
    {},
    cliProgress.Presets.shades_classic,
  );
  bar.start(options.count, 0);
 
  const report: DynamicBenchmarker.IReport = await DynamicBenchmarker.master({
    servant: `${__dirname}/servant.js`,
    count: options.count,
    threads: options.threads,
    simultaneous: options.simultaneous,
    filter: (func) =>
      (!options.include?.length ||
        (options.include ?? []).some((str) => func.includes(str))) &&
      (!options.exclude?.length ||
        (options.exclude ?? []).every((str) => !func.includes(str))),
    progress: (value: number) => {
      if (value >= 100 + prev.value) {
        bar.update(value);
        prev.value = value;
      }
    },
    stdio: "ignore",
  });
  bar.stop();
 
  // DOCUMENTATION
  try {
    await fs.promises.mkdir(`${MyConfiguration.ROOT}/docs/benchmarks`, {
      recursive: true,
    });
  } catch {}
  await fs.promises.writeFile(
    `${MyConfiguration.ROOT}/docs/benchmarks/${os
      .cpus()[0]
      .model.trim()
      .split("\\")
      .join("")
      .split("/")
      .join("")}.md`,
    DynamicBenchmarker.markdown(report),
    "utf8",
  );
 
  // CLOSE
  await backend.close();
};
main().catch((exp) => {
  console.error(exp);
  process.exit(-1);
});

To compose the benchmark program of @nestia/benchmark on your backend application, you have to create two executable TypeScript programs; the main program and the servant program.

The main program is executed by user (npm run benchmark command in the playground project), and centralizes the benchmark progress. It creates multiple servant programs parallel, and aggregate the benchmark results from them. After the aggregation, it publishes the benchmark report with markdown format.

The servant program is executed by the main program multiply in parallel, and actually runs the e2e test functions for benchmarking. Composing the servant program, you have to specify the directory where the e2e test functions are located. Also, composing the main program of benchmark, you also have to specify the file location of the servant program.

If you want to see more benchmark program cases, visit below links:

Project	Main	Servant	Report
`samchon/nestia-start`	index.ts	servant.ts	REPORT.md
`samchon/backend`	index.ts	servant.ts	REPORT.md

Test Functions

test/features/api/bbs/test_api_bbs_article_create.ts

import { RandomGenerator, TestValidator } from "@nestia/e2e";
import { v4 } from "uuid";
 
import api from "@ORGANIZATION/PROJECT-api/lib/index";
import { IBbsArticle } from "@ORGANIZATION/PROJECT-api/lib/structures/bbs/IBbsArticle";
 
export async function test_api_bbs_article_create(
  connection: api.IConnection,
): Promise<void> {
  // STORE A NEW ARTICLE
  const stored: IBbsArticle = await api.functional.bbs.articles.create(
    connection,
    "general",
    {
      writer: RandomGenerator.name(),
      title: RandomGenerator.paragraph(3)(),
      body: RandomGenerator.content(8)()(),
      format: "txt",
      files: [
        {
          name: "logo",
          extension: "png",
          url: "https://somewhere.com/logo.png",
        },
      ],
      password: v4(),
    },
  );
 
  // READ THE DATA AGAIN
  const read: IBbsArticle = await api.functional.bbs.articles.at(
    connection,
    stored.section,
    stored.id,
  );
  TestValidator.equals("created")(stored)(read);
}

test/features/api/bbs/test_api_bbs_article_update.ts

import { RandomGenerator, TestValidator } from "@nestia/e2e";
import { v4 } from "uuid";
 
import api from "@ORGANIZATION/PROJECT-api/lib/index";
import { IBbsArticle } from "@ORGANIZATION/PROJECT-api/lib/structures/bbs/IBbsArticle";
 
export async function test_api_bbs_article_update(
  connection: api.IConnection,
): Promise<void> {
  // STORE A NEW ARTICLE
  const password: string = v4();
  const article: IBbsArticle = await api.functional.bbs.articles.create(
    connection,
    "general",
    {
      writer: RandomGenerator.name(),
      title: RandomGenerator.paragraph(3)(),
      body: RandomGenerator.content(8)()(),
      format: "txt",
      files: [
        {
          name: "logo",
          extension: "png",
          url: "https://somewhere.com/logo.png",
        },
      ],
      password,
    },
  );
 
  // UPDATE WITH EXACT PASSWORD
  const content: IBbsArticle.ISnapshot =
    await api.functional.bbs.articles.update(
      connection,
      article.section,
      article.id,
      {
        title: RandomGenerator.paragraph(3)(),
        body: RandomGenerator.content(8)()(),
        format: "txt",
        files: [],
        password,
      },
    );
  article.snapshots.push(content);
 
  // TRY UPDATE WITH WRONG PASSWORD
  await TestValidator.error("update with wrong password")(() =>
    api.functional.bbs.articles.update(
      connection,
      article.section,
      article.id,
      {
        title: RandomGenerator.paragraph(5)(),
        body: RandomGenerator.content(8)()(),
        format: "txt",
        files: [],
        password: v4(),
      },
    ),
  );
}

test/features/api/bbs/test_api_bbs_article_index_search.ts

import { ArrayUtil, RandomGenerator, TestValidator } from "@nestia/e2e";
 
import api from "@ORGANIZATION/PROJECT-api/lib/index";
import { IBbsArticle } from "@ORGANIZATION/PROJECT-api/lib/structures/bbs/IBbsArticle";
import { IPage } from "@ORGANIZATION/PROJECT-api/lib/structures/common/IPage";
 
export async function test_api_bbs_article_index_search(
  connection: api.IConnection,
): Promise<void> {
  // GENERATE 100 ARTICLES
  const section: string = "general";
  const articles: IBbsArticle[] = await ArrayUtil.asyncRepeat(100)(() =>
    api.functional.bbs.articles.create(connection, section, {
      writer: RandomGenerator.name(),
      title: RandomGenerator.paragraph(4)(),
      body: RandomGenerator.content(3)()(),
      format: "txt",
      files: [],
      password: RandomGenerator.alphabets(8),
    }),
  );
 
  // GET ENTIRE DATA
  const total: IPage<IBbsArticle.ISummary> =
    await api.functional.bbs.articles.index(connection, section, {
      limit: articles.length,
      sort: ["-created_at"],
    });
 
  // PREPARE SEARCH FUNCTION
  const search = TestValidator.search("BbsArticleProvider.index()")(
    async (input: IBbsArticle.IRequest.ISearch) => {
      const page: IPage<IBbsArticle.ISummary> =
        await api.functional.bbs.articles.index(connection, section, {
          limit: articles.length,
          search: input,
          sort: ["-created_at"],
        });
      return page.data;
    },
  )(total.data, 10);
 
  // SEARCH TITLE
  await search({
    fields: ["title"],
    values: (article) => [article.title],
    request: ([title]) => ({ title }),
    filter: (article, [title]) => article.title.includes(title),
  });
 
  // SEARCH WRITER
  await search({
    fields: ["writer"],
    values: (article) => [article.writer],
    request: ([writer]) => ({ writer }),
    filter: (article, [writer]) => article.writer.includes(writer),
  });
 
  // SEARCH BOTH OF THEM
  await search({
    fields: ["title", "writer"],
    values: (article) => [article.title, article.writer],
    request: ([title, writer]) => ({ title, writer }),
    filter: (article, [title, writer]) =>
      article.title.includes(title) && article.writer.includes(writer),
  });
}

Developing e2e test functions are very easy. Just make e2e based test function utilizing @nestia/sdk generated SDK library, and exports the function with test_ prefixed name (If you’ve configured another prefix property in the benchmark main program, just follow the configuration).

Also, make the function to have parameter(s) configured in the servant program of the benchmark. As above test functions are examples of playground project that has configured to have only one connection parameter, All of them have the only one parameter connection.

After composing these e2e test functions, just execute the benchmark main program. In the playground project, it can be exeucted by npm run benchmark command. The benchmark program will run these e2e test functions in parallel and randomly, and measure the performance of your backend server.

git clone https://github.com/samchon/nestia-start
cd nestia-start
npm install
npm run build:test
npm run benchmark

Test Program Develop Migration from Swagger