New Issue

[Bug]: Native-Utils is now a "Security Placeholder" Package #9797

Closed

opened by daikei · 13 comments

daikei commented

What happened?

Attempting to build a new version of Calckey through docker only results in the backend failing to start. Looking at the package, the package was replaced with a security advisory.

What did you expect to happen?

No response

Version

Docker/13.2.0-Beta, Docker/RC

Instance

No response

What browser are you using?

Firefox

Relevant log output

# Security holding package

This package contained malicious code and was removed from the registry by the npm security team. A placeholder was published to ensure users are not affected in the future.

Please refer to www.npmjs.com/advisories?search=native-utils for more information.

Contribution Guidelines

I agree to follow this project's Contribution Guidelines

### What happened? Attempting to build a new version of Calckey through docker only results in the backend failing to start. Looking at the package, the package was replaced with a security advisory. ### What did you expect to happen? _No response_ ### Version Docker/13.2.0-Beta, Docker/RC ### Instance _No response_ ### What browser are you using? Firefox ### Relevant log output ```shell # Security holding package This package contained malicious code and was removed from the registry by the npm security team. A placeholder was published to ensure users are not affected in the future. Please refer to www.npmjs.com/advisories?search=native-utils for more information. ``` ### Contribution Guidelines - [x] I agree to follow this project's Contribution Guidelines

thatonecalculator added the

🔥high priority

🐛Bug

labels

thatonecalculator commented

Owner

As per packages/backend/package.json:

		"native-utils": "link:native-utils",

should be loaded locally given link:

As per `packages/backend/package.json`: ``` "native-utils": "link:native-utils", ``` should be loaded locally given `link:`

thatonecalculator commented

Owner

cc @s1idewhist1e @cutestnekoaqua

s1idewhist1e commented

Collaborator

I'm not sure why this is happening, it shohld be loaded locally. I'll look into it soon.

s1idewhist1e commented

Collaborator

@thatonecalculator this happens only in docker so I can't help

s1idewhist1e commented

Collaborator

I have a theory that it's because the dockerfile doesn't copy over the local package before running pnpm i but I'm not familiar enough with docker to say for sure

# Copy only the dependency-related files first, to cache efficiently
COPY package.json pnpm*.yaml ./
COPY packages/backend/package.json packages/backend/package.json
COPY packages/client/package.json packages/client/package.json
COPY packages/sw/package.json packages/sw/package.json

# Configure corepack and pnpm
RUN corepack enable
RUN corepack prepare pnpm@latest --activate

# Install dev mode dependencies for compilation
RUN pnpm i --frozen-lockfile

# Copy in the rest of the files, to compile from TS to JS
COPY . ./
RUN pnpm run build

I have a theory that it's because the dockerfile doesn't copy over the local package before running `pnpm i` but I'm not familiar enough with docker to say for sure ```docker # Copy only the dependency-related files first, to cache efficiently COPY package.json pnpm*.yaml ./ COPY packages/backend/package.json packages/backend/package.json COPY packages/client/package.json packages/client/package.json COPY packages/sw/package.json packages/sw/package.json # Configure corepack and pnpm RUN corepack enable RUN corepack prepare pnpm@latest --activate # Install dev mode dependencies for compilation RUN pnpm i --frozen-lockfile # Copy in the rest of the files, to compile from TS to JS COPY . ./ RUN pnpm run build ```

thatonecalculator commented

Owner

I'm not the best with docker either...

thatonecalculator referenced this issue from a commit

attempt to fix #9797

thatonecalculator commented

Owner

@daikei I just pushed a commit to dev -- can you pull and try it out?

daikei commented

Poster

creating an image using bf44a0a719

$ sudo docker image build .
[+] Building 371.5s (30/30) FINISHED
 => [internal] load build definition from Dockerfile 0.0s
 => => transferring dockerfile: 1.97kB 0.0s
 => [internal] load .dockerignore 0.0s
 => => transferring context: 655B 0.0s
 => [internal] load metadata for docker.io/library/node:19-alpine 1.1s
 => [internal] load build context 0.3s
 => => transferring context: 35.37MB 0.3s
 => [stage-1  1/12] FROM docker.io/library/node:19-alpine@sha256:ce9dd01fefe302dce994c07fb32b55403a79acd693c9d841571dc14fc47826c8                                                                                                       0.1s
 => => resolve docker.io/library/node:19-alpine@sha256:ce9dd01fefe302dce994c07fb32b55403a79acd693c9d841571dc14fc47826c8                                                                                                                 0.0s
 => => sha256:31f511a1df6629e0853968af162a1ea55a80fe07d35a1d1c96a5bca736823097 1.16kB / 1.16kB                                                                                                                                          0.0s
 => => sha256:36722213e3daf1be0a079f6de80644a5d6651b133e0f3ecaaf1593f858170817 6.48kB / 6.48kB                                                                                                                                          0.0s
 => => sha256:ce9dd01fefe302dce994c07fb32b55403a79acd693c9d841571dc14fc47826c8 1.43kB / 1.43kB                                                                                                                                          0.0s
 => [stage-1  2/12] WORKDIR /calckey 0.0s
 => [build  3/15] RUN apk add --no-cache --no-progress git alpine-sdk python3 rust cargo 11.0s
 => [stage-1  3/12] RUN apk add --no-cache --no-progress tini ffmpeg vips-dev 11.3s
 => [build  4/15] COPY package.json pnpm*.yaml ./ 0.0s
 => [build  5/15] COPY packages/backend/package.json packages/backend/package.json 0.0s
 => [build  6/15] COPY packages/client/package.json packages/client/package.json 0.0s
 => [build  7/15] COPY packages/sw/package.json packages/sw/package.json 0.0s
 => [build  8/15] COPY packages/backend/native-utils/package.json packages/backend/native-utils/package.json 0.0s
 => [build  9/15] COPY packages/backend/native-utils/**/*/package.json packages/backend/native-utils/**/*/package.json 0.0s
 => [build 10/15] RUN corepack enable 0.5s 
 => [stage-1  4/12] COPY . ./ 0.2s 
 => [build 11/15] RUN corepack prepare pnpm@latest --activate 2.0s 
 => [build 12/15] RUN pnpm i --frozen-lockfile 172.5s 
 => [build 13/15] COPY . ./ 0.2s 
 => [build 14/15] RUN pnpm run build 151.4s 
 => [build 15/15] RUN pnpm i --prod --frozen-lockfile 2.8s 
 => [stage-1  5/12] COPY --from=build /calckey/node_modules /calckey/node_modules 8.3s 
 => [stage-1  6/12] COPY --from=build /calckey/packages/backend/node_modules /calckey/packages/backend/node_modules 0.0s 
 => [stage-1  7/12] COPY --from=build /calckey/packages/sw/node_modules /calckey/packages/sw/node_modules 0.0s 
 => [stage-1  8/12] COPY --from=build /calckey/packages/client/node_modules /calckey/packages/client/node_modules 0.0s 
 => [stage-1  9/12] COPY --from=build /calckey/built /calckey/built 0.2s 
 => [stage-1 10/12] COPY --from=build /calckey/packages/backend/built /calckey/packages/backend/built 0.1s 
 => [stage-1 11/12] COPY --from=build /calckey/packages/backend/assets/instance.css /calckey/packages/backend/assets/instance.css 0.0s
 => [stage-1 12/12] RUN corepack enable 0.3s
 => exporting to image 9.5s
 => => exporting layers 9.5s
 => => writing image sha256:d70e70792adb4c65093bef1de48a2ca8d741c0563f63ec63f46f4f579e0022a7

resulted in the following error:

   ___      _      _              
  / __\__ _| | ___| | _____ _   _ 
 / /  / _` | |/ __| |/ / _  | | |
/ /__| (_| | | (__|   <  __/ |_| |
\____/\__,_|_|\___|_|\_\___|\__, |
                            (___/ 
 Calckey is an open-source decentralized microblogging platform.
 If you like Calckey, please consider starring or contributing to the repo. https://codeberg.org/calckey/calckey
--- 7dbc6e53fb90 (PID: 90) ---
INFO *	[core boot]	Welcome to Calckey!
INFO *	[core boot]	Calckey v13.2.0-dev22
INFO *	[core boot env]	NODE_ENV: production
INFO *	[core boot nodejs]	Version v19.8.1 detected.
DONE *	[core boot config]	Loaded
INFO *	[core boot db]	Connecting...
DONE *	[core boot db]	Connected: v15.2 (Debian 15.2-1.pgdg110+1)
DONE *	[core boot]	Calckey initialized
INFO *	[core boot]	Starting 1 worker...
Error [ERR_MODULE_NOT_FOUND]: Cannot find package '/calckey/packages/backend/node_modules/native-utils/' imported from /calckey/packages/backend/built/server/api/index.js
    at new NodeError (node:internal/errors:399:5)
    at legacyMainResolve (node:internal/modules/esm/resolve:202:9)
    at packageResolve (node:internal/modules/esm/resolve:781:14)
    at moduleResolve (node:internal/modules/esm/resolve:843:20)
    at defaultResolve (node:internal/modules/esm/resolve:1058:11)
    at nextResolve (node:internal/modules/esm/hooks:654:28)
    at Hooks.resolve (node:internal/modules/esm/hooks:309:30)
    at ESMLoader.resolve (node:internal/modules/esm/loader:312:26)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:172:38)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:76:40)
    at link (node:internal/modules/esm/module_job:75:36) {
  code: 'ERR_MODULE_NOT_FOUND'
}
(node:101) ExperimentalWarning: Import assertions are not a stable feature of the JavaScript language. Avoid relying on their current behavior and syntax as those might change in a future version of Node.js.
(Use `node --trace-warnings ...` to show where the warning was created)
(node:101) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time

creating an image using bf44a0a719a9831e21e3530a7ee8dd716d597d61 ``` $ sudo docker image build . [+] Building 371.5s (30/30) FINISHED => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 1.97kB 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 655B 0.0s => [internal] load metadata for docker.io/library/node:19-alpine 1.1s => [internal] load build context 0.3s => => transferring context: 35.37MB 0.3s => [stage-1 1/12] FROM docker.io/library/node:19-alpine@sha256:ce9dd01fefe302dce994c07fb32b55403a79acd693c9d841571dc14fc47826c8 0.1s => => resolve docker.io/library/node:19-alpine@sha256:ce9dd01fefe302dce994c07fb32b55403a79acd693c9d841571dc14fc47826c8 0.0s => => sha256:31f511a1df6629e0853968af162a1ea55a80fe07d35a1d1c96a5bca736823097 1.16kB / 1.16kB 0.0s => => sha256:36722213e3daf1be0a079f6de80644a5d6651b133e0f3ecaaf1593f858170817 6.48kB / 6.48kB 0.0s => => sha256:ce9dd01fefe302dce994c07fb32b55403a79acd693c9d841571dc14fc47826c8 1.43kB / 1.43kB 0.0s => [stage-1 2/12] WORKDIR /calckey 0.0s => [build 3/15] RUN apk add --no-cache --no-progress git alpine-sdk python3 rust cargo 11.0s => [stage-1 3/12] RUN apk add --no-cache --no-progress tini ffmpeg vips-dev 11.3s => [build 4/15] COPY package.json pnpm*.yaml ./ 0.0s => [build 5/15] COPY packages/backend/package.json packages/backend/package.json 0.0s => [build 6/15] COPY packages/client/package.json packages/client/package.json 0.0s => [build 7/15] COPY packages/sw/package.json packages/sw/package.json 0.0s => [build 8/15] COPY packages/backend/native-utils/package.json packages/backend/native-utils/package.json 0.0s => [build 9/15] COPY packages/backend/native-utils/**/*/package.json packages/backend/native-utils/**/*/package.json 0.0s => [build 10/15] RUN corepack enable 0.5s => [stage-1 4/12] COPY . ./ 0.2s => [build 11/15] RUN corepack prepare pnpm@latest --activate 2.0s => [build 12/15] RUN pnpm i --frozen-lockfile 172.5s => [build 13/15] COPY . ./ 0.2s => [build 14/15] RUN pnpm run build 151.4s => [build 15/15] RUN pnpm i --prod --frozen-lockfile 2.8s => [stage-1 5/12] COPY --from=build /calckey/node_modules /calckey/node_modules 8.3s => [stage-1 6/12] COPY --from=build /calckey/packages/backend/node_modules /calckey/packages/backend/node_modules 0.0s => [stage-1 7/12] COPY --from=build /calckey/packages/sw/node_modules /calckey/packages/sw/node_modules 0.0s => [stage-1 8/12] COPY --from=build /calckey/packages/client/node_modules /calckey/packages/client/node_modules 0.0s => [stage-1 9/12] COPY --from=build /calckey/built /calckey/built 0.2s => [stage-1 10/12] COPY --from=build /calckey/packages/backend/built /calckey/packages/backend/built 0.1s => [stage-1 11/12] COPY --from=build /calckey/packages/backend/assets/instance.css /calckey/packages/backend/assets/instance.css 0.0s => [stage-1 12/12] RUN corepack enable 0.3s => exporting to image 9.5s => => exporting layers 9.5s => => writing image sha256:d70e70792adb4c65093bef1de48a2ca8d741c0563f63ec63f46f4f579e0022a7 ``` resulted in the following error: ``` ___ _ _ / __\__ _| | ___| | _____ _ _ / / / _` | |/ __| |/ / _ | | | / /__| (_| | | (__| < __/ |_| | \____/\__,_|_|\___|_|\_\___|\__, | (___/ Calckey is an open-source decentralized microblogging platform. If you like Calckey, please consider starring or contributing to the repo. https://codeberg.org/calckey/calckey --- 7dbc6e53fb90 (PID: 90) --- INFO * [core boot] Welcome to Calckey! INFO * [core boot] Calckey v13.2.0-dev22 INFO * [core boot env] NODE_ENV: production INFO * [core boot nodejs] Version v19.8.1 detected. DONE * [core boot config] Loaded INFO * [core boot db] Connecting... DONE * [core boot db] Connected: v15.2 (Debian 15.2-1.pgdg110+1) DONE * [core boot] Calckey initialized INFO * [core boot] Starting 1 worker... Error [ERR_MODULE_NOT_FOUND]: Cannot find package '/calckey/packages/backend/node_modules/native-utils/' imported from /calckey/packages/backend/built/server/api/index.js at new NodeError (node:internal/errors:399:5) at legacyMainResolve (node:internal/modules/esm/resolve:202:9) at packageResolve (node:internal/modules/esm/resolve:781:14) at moduleResolve (node:internal/modules/esm/resolve:843:20) at defaultResolve (node:internal/modules/esm/resolve:1058:11) at nextResolve (node:internal/modules/esm/hooks:654:28) at Hooks.resolve (node:internal/modules/esm/hooks:309:30) at ESMLoader.resolve (node:internal/modules/esm/loader:312:26) at ESMLoader.getModuleJob (node:internal/modules/esm/loader:172:38) at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:76:40) at link (node:internal/modules/esm/module_job:75:36) { code: 'ERR_MODULE_NOT_FOUND' } (node:101) ExperimentalWarning: Import assertions are not a stable feature of the JavaScript language. Avoid relying on their current behavior and syntax as those might change in a future version of Node.js. (Use `node --trace-warnings ...` to show where the warning was created) (node:101) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time ```

daikei commented

Poster

I'm just smashing rocks together, but perhaps it's an idea to move the native-utils into node_modules? It's sitting in root/packages/backend/native-utils