-
Notifications
You must be signed in to change notification settings - Fork 45
Description
After 2 days of fighting the system back and forth, I just arrived at the conclusion that either plrust cannot work in a Docker environment or the docs fail to explain some necessary steps.
So I leave here my attempt to install plrust from scratch using a Dockerfile, along with the errors I experience.
ARG PG_VERSION_MAJOR=16
FROM postgres:${PG_VERSION_MAJOR} AS builder
ARG PG_VERSION_MAJOR
ENV PG_VERSION_MAJOR=${PG_VERSION_MAJOR}
RUN apt-get update && apt-get install -y --no-install-recommends --no-install-suggests \
bison \
build-essential \
ca-certificates \
ccache \
curl \
crossbuild-essential-arm64 \
crossbuild-essential-amd64 \
flex \
clang \
gcc \
git \
gnupg \
libcurl4-openssl-dev \
libicu72 \
libopenblas-dev \
libreadline-dev \
libssl-dev \
libxml2-dev \
libxml2-utils \
libxslt-dev \
lsb-release \
locales \
make \
pkg-config \
postgresql-server-dev-${PG_VERSION_MAJOR} \
software-properties-common \
xsltproc \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*
# PRE-REQUISITES
# ===============
# 1. Permissions
USER root
RUN chown postgres -R /usr/share/postgresql/${PG_VERSION_MAJOR}/extension/
RUN chown postgres -R /usr/lib/postgresql/${PG_VERSION_MAJOR}/lib/
USER postgres
# 2. Install rust
# RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain "stable" -y
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain "1.72.0" -y
ENV PATH="/var/lib/postgresql/.cargo/bin:$PATH"
# 3. Clone plrust and check Rust version
WORKDIR /tmp
RUN git clone https://github.com/tcdi/plrust.git
WORKDIR /tmp/plrust/plrust
RUN rustup default 1.72.0
# 4. Install pgrx
# I found that if pgrx version doesn't match the one in Cargo.toml, the compilation will fail
ENV PGX_HOME=/usr/lib/postgresql/${PG_VERSION_MAJOR}
RUN PGRX_VERSION=$(sed -n 's/^pgrx *=[^0-9]*\([0-9.]*\)".*/\1/p' Cargo.toml) && \
echo "PGRX_VERSION=$PGRX_VERSION" && \
cargo install cargo-pgrx --locked --force --version "${PGRX_VERSION}"
RUN cargo pgrx init --pg${PG_VERSION_MAJOR} /usr/lib/postgresql/${PG_VERSION_MAJOR}/bin/pg_config
# INSTALL PL/RUST
# ===============
# 5. Install trusted plrust with cross-compilation
RUN rustup component add llvm-tools-preview rustc-dev
# we are in debian, so I pick `...unknown-linux-gnu`
RUN rustup target install aarch64-unknown-linux-gnu
RUN rustup target install x86_64-unknown-linux-gnu
WORKDIR /tmp/plrust/plrustc
RUN ./build.sh
WORKDIR /tmp/plrust/build/bin/
RUN cp plrustc /var/lib/postgresql/.cargo/bin/
WORKDIR /tmp/plrust/plrust
RUN PG_VER=${PG_VERSION_MAJOR} STD_TARGETS="aarch64-postgres-linux-gnu " ./build
RUN cargo pgrx install --release --features trusted --pg-config /usr/lib/postgresql/${PG_VERSION_MAJOR}/bin/pg_config
ENV SCRATCH_DIR="/var/lib/postgresql/plrust-scratch"
RUN mkdir -p ${SCRATCH_DIR}
WORKDIR /tmp/plrust/plrust
# 6. Reset permissions
USER root
RUN chown root -R /usr/share/postgresql/${PG_VERSION_MAJOR}/extension/
RUN chown root -R /usr/lib/postgresql/${PG_VERSION_MAJOR}/lib/
USER postgres
# 7. Run with shared_preload_libraries=plrust + plrust.work_dir
CMD ["postgres", "-c", "shared_preload_libraries=plrust", "-c", "plrust.work_dir='/var/lib/postgresql/plrust-scratch'", "-c", "logging_collector=on"]After building this Dockefile + spawning the corresponding container, I can enter into a psql session, where I try to do:
CREATE EXTENSION plrust;
-- returns `CREATE EXTENSION` without "untrusted" warning or errorthen
CREATE FUNCTION add_two_numbers(a NUMERIC, b NUMERIC) RETURNS NUMERIC STRICT LANGUAGE plrust AS $$
Ok(Some(a + b))
$$;gives this log:
ERROR:
0: `cargo build` failed
Location:
/rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/convert/mod.rs:716
`cargo build` stderr:
Updating crates.io index
Locking 147 packages to latest compatible versions
Adding pgrx v0.11.0 (available: v0.11.4)
Adding pgrx-macros v0.11.0 (available: v0.11.4)
Adding pgrx-pg-config v0.11.0 (available: v0.11.4)
Adding pgrx-pg-sys v0.11.0 (available: v0.11.4)
Adding pgrx-sql-entity-graph v0.11.0 (available: v0.11.4)
Downloading crates ...
Downloaded plrust-trusted-pgrx v1.2.8
Downloaded bitflags v2.6.0
Downloaded write16 v1.0.0
Downloaded zerofrom v0.1.5
Downloaded yoke-derive v0.7.5
Downloaded enum-map-derive v0.17.0
Downloaded tinystr v0.7.6
Downloaded utf8_iter v1.0.4
Downloaded serde_spanned v0.6.8
Downloaded memoffset v0.9.1
Downloaded lazy_static v1.5.0
Downloaded shlex v1.3.0
Downloaded zerovec-derive v0.10.3
Downloaded yoke v0.7.5
Downloaded toml_datetime v0.6.8
Downloaded getrandom v0.2.15
Downloaded semver-parser v0.10.3
Downloaded quote v1.0.37
Downloaded thiserror v1.0.69
Downloaded uuid v1.11.0
Downloaded unicode-ident v1.0.14
Downloaded ucd-trie v0.1.7
Downloaded serde_derive v1.0.216
Downloaded url v2.5.4
Downloaded toml_edit v0.22.22
Downloaded unicode-segmentation v1.12.0
Downloaded zerovec v0.10.4
Downloaded winnow v0.6.20
Downloaded hashbrown v0.15.2
Downloaded serde_json v1.0.133
Downloaded idna v1.0.3
Downloaded icu_properties_data v1.5.0
Downloaded aho-corasick v1.1.3
Downloaded regex v1.11.1
Downloaded pest v2.7.15
Downloaded syn v2.0.90
Downloaded memchr v2.7.4
Downloaded regex-syntax v0.8.5
Downloaded indexmap v2.7.0
Downloaded icu_collections v1.5.0
Downloaded serde v1.0.216
Downloaded icu_properties v1.5.1
Downloaded heapless v0.7.17
Downloaded proc-macro2 v1.0.92
Downloaded icu_provider v1.5.0
Downloaded icu_normalizer_data v1.5.0
Downloaded icu_normalizer v1.5.0
Downloaded icu_locid v1.5.0
Downloaded eyre v0.6.12
Downloaded anyhow v1.0.94
Downloaded toml v0.8.19
Downloaded thiserror-impl v1.0.69
Downloaded ryu v1.0.18
Downloaded icu_locid_transform_data v1.5.0
Downloaded half v1.8.3
Downloaded regex-automata v0.4.9
Downloaded libc v0.2.168
Downloaded semver v1.0.23
Downloaded libloading v0.8.6
Downloaded form_urlencoded v1.2.1
Downloaded petgraph v0.6.5
Downloaded thiserror v2.0.6
Downloaded smallvec v1.13.2
Downloaded once_cell v1.20.2
Downloaded clang-sys v1.8.1
Downloaded litemap v0.7.4
Downloaded icu_locid_transform v1.5.0
Downloaded zerofrom-derive v0.1.5
Downloaded writeable v0.5.5
Downloaded utf16_iter v1.0.5
Downloaded thiserror-impl v2.0.6
Downloaded synstructure v0.13.1
Downloaded rustc_version v0.4.1
Downloaded percent-encoding v2.3.1
Downloaded itoa v1.0.14
Downloaded idna_adapter v1.2.0
Downloaded enum-map v2.7.3
Downloaded displaydoc v0.2.5
Downloaded icu_provider_macros v1.5.0
Downloaded autocfg v1.4.0
Compiling proc-macro2 v1.0.92
Compiling unicode-ident v1.0.14
Compiling serde v1.0.216
Compiling stable_deref_trait v1.2.0
error: the `-Z unstable-options` flag must also be passed to enable the flag `check-cfg`
error: could not compile `proc-macro2` (build script)
Caused by:
process didn't exit successfully: `plrustc --crate-name build_script_build --edition=2021 /var/lib/postgresql/.cargo/registry/src/index.crates.io-6f17d22bba15001f/proc-macro2-1.0.92/build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg 'feature="default"' --cfg 'feature="proc-macro"' --cfg 'feature="span-locations"' --check-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values("default", "nightly", "proc-macro", "span-locations"))' -C metadata=e8c0aefce17821ba -C extra-filename=-e8c0aefce17821ba --out-dir '/var/lib/postgresql/data/'\''/var/lib/postgresql/plrust-scratch'\''/plrust_fn_oid_5_16394_3191160700928/'\''/var/lib/postgresql/plrust-scratch'\''/target/release/build/proc-macro2-e8c0aefce17821ba' -C strip=debuginfo -L 'dependency=/var/lib/postgresql/data/'\''/var/lib/postgresql/plrust-scratch'\''/plrust_fn_oid_5_16394_3191160700928/'\''/var/lib/postgresql/plrust-scratch'\''/target/release/deps' --cap-lints allow` (exit status: 1)
warning: build failed, waiting for other jobs to finish...
error: the `-Z unstable-options` flag must also be passed to enable the flag `check-cfg`
error: could not compile `unicode-ident` (lib)
Caused by:
process didn't exit successfully: `plrustc --crate-name unicode_ident --edition=2018 /var/lib/postgresql/.cargo/registry/src/index.crates.io-6f17d22bba15001f/unicode-ident-1.0.14/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --check-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values())' -C metadata=0fc4f3e53070b983 -C extra-filename=-0fc4f3e53070b983 --out-dir '/var/lib/postgresql/data/'\''/var/lib/postgresql/plrust-scratch'\''/plrust_fn_oid_5_16394_3191160700928/'\''/var/lib/postgresql/plrust-scratch'\''/target/release/deps' -C strip=debuginfo -L 'dependency=/var/lib/postgresql/data/'\''/var/lib/postgresql/plrust-scratch'\''/plrust_fn_oid_5_16394_3191160700928/'\''/var/lib/postgresql/plrust-scratch'\''/target/release/deps' --cap-lints allow` (exit status: 1)
error: the `-Z unstable-options` flag must also be passed to enable the flag `check-cfg`
error: could not compile `stable_deref_trait` (lib)
Caused by:
process didn't exit successfully: `plrustc --crate-name stable_deref_trait --edition=2015 /var/lib/postgresql/.cargo/registry/src/index.crates.io-6f17d22bba15001f/stable_deref_trait-1.2.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg 'feature="alloc"' --check-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values("alloc", "default", "std"))' -C metadata=276654b4846bbedc -C extra-filename=-276654b4846bbedc --out-dir '/var/lib/postgresql/data/'\''/var/lib/postgresql/plrust-scratch'\''/plrust_fn_oid_5_16394_3191160700928/'\''/var/lib/postgresql/plrust-scratch'\''/target/release/deps' -C strip=debuginfo -L 'dependency=/var/lib/postgresql/data/'\''/var/lib/postgresql/plrust-scratch'\''/plrust_fn_oid_5_16394_3191160700928/'\''/var/lib/postgresql/plrust-scratch'\''/target/release/deps' --cap-lints allow` (exit status: 1)
error: the `-Z unstable-options` flag must also be passed to enable the flag `check-cfg`
error: could not compile `serde` (build script)
Caused by:
process didn't exit successfully: `plrustc --crate-name build_script_build --edition=2018 /var/lib/postgresql/.cargo/registry/src/index.crates.io-6f17d22bba15001f/serde-1.0.216/build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg 'feature="default"' --cfg 'feature="derive"' --cfg 'feature="serde_derive"' --cfg 'feature="std"' --check-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values("alloc", "default", "derive", "rc", "serde_derive", "std", "unstable"))' -C metadata=c2ae0e5c09875656 -C extra-filename=-c2ae0e5c09875656 --out-dir '/var/lib/postgresql/data/'\''/var/lib/postgresql/plrust-scratch'\''/plrust_fn_oid_5_16394_3191160700928/'\''/var/lib/postgresql/plrust-scratch'\''/target/release/build/serde-c2ae0e5c09875656' -C strip=debuginfo -L 'dependency=/var/lib/postgresql/data/'\''/var/lib/postgresql/plrust-scratch'\''/plrust_fn_oid_5_16394_3191160700928/'\''/var/lib/postgresql/plrust-scratch'\''/target/release/deps' --cap-lints allow` (exit status: 1)
Source Code:
#![deny(unsafe_op_in_unsafe_fn)]
pub mod opened {
#[allow(unused_imports)]
use pgrx::prelude::*;
#[allow(unused_lifetimes)]
#[pg_extern]
fn plrust_fn_oid_5_16394<'a>(
a: pgrx::AnyNumeric,
b: pgrx::AnyNumeric,
) -> ::std::result::Result<
Option<pgrx::AnyNumeric>,
Box<dyn std::error::Error + Send + Sync + 'static>,
> {
Ok(Some(a + b))
}
}
#[deny(unknown_lints)]
mod forbidden {
#![forbid(deprecated)]
#![forbid(implied_bounds_entailment)]
#![forbid(plrust_async)]
#![forbid(plrust_autotrait_impls)]
#![forbid(plrust_closure_trait_impl)]
#![forbid(plrust_env_macros)]
#![forbid(plrust_extern_blocks)]
#![forbid(plrust_external_mod)]
#![forbid(plrust_filesystem_macros)]
#![forbid(plrust_fn_pointers)]
#![forbid(plrust_leaky)]
#![forbid(plrust_lifetime_parameterized_traits)]
#![forbid(plrust_print_macros)]
#![forbid(plrust_static_impls)]
#![forbid(plrust_stdio)]
#![forbid(plrust_suspicious_trait_object)]
#![forbid(soft_unstable)]
#![forbid(suspicious_auto_trait_impls)]
#![forbid(unsafe_code)]
#![forbid(where_clauses_object_safety)]
#[allow(unused_imports)]
use pgrx::prelude::*;
#[allow(unused_lifetimes)]
fn plrust_fn_oid_5_16394<'a>(
a: pgrx::AnyNumeric,
b: pgrx::AnyNumeric,
) -> ::std::result::Result<
Option<pgrx::AnyNumeric>,
Box<dyn std::error::Error + Send + Sync + 'static>,
> {
Ok(Some(a + b))
}
}
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.I also tried to move things around here by following the Dockerfile.try in the project:
ARG PG_VERSION_MAJOR=16
FROM postgres:${PG_VERSION_MAJOR} AS builder
ARG PG_VERSION_MAJOR
ENV PG_VERSION_MAJOR=${PG_VERSION_MAJOR}
RUN apt-get update && apt-get install -y --no-install-recommends --no-install-suggests \
bison \
build-essential \
ca-certificates \
ccache \
curl \
crossbuild-essential-arm64 \
crossbuild-essential-amd64 \
flex \
clang \
gcc \
git \
gnupg \
libcurl4-openssl-dev \
libicu72 \
libopenblas-dev \
libreadline-dev \
libssl-dev \
libxml2-dev \
libxml2-utils \
libxslt-dev \
lsb-release \
locales \
make \
pkg-config \
postgresql-server-dev-${PG_VERSION_MAJOR} \
software-properties-common \
xsltproc \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*
# PRE-REQUISITES
# 1. Permissions
USER root
RUN chown postgres -R /usr/share/postgresql/${PG_VERSION_MAJOR}/extension/
RUN chown postgres -R /usr/lib/postgresql/${PG_VERSION_MAJOR}/lib/
USER postgres
# 2. Install rust
# RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain "stable" -y
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain "1.72.0"
ENV PATH="/var/lib/postgresql/.cargo/bin:$PATH"
# 3. Clone plrust and check Rust version
WORKDIR /tmp
RUN git clone https://github.com/tcdi/plrust.git
WORKDIR /tmp/plrust/plrust
# Switch rustup to match pl_rust's version overriden by /tmp/plrust/rust-toolchain.toml, which we can get easily by running `rustc -V` from inside the plrust/plrust directory
# RUN rustup default $(rustc -V | cut -d' ' -f2)
RUN rustup toolchain install 1.72.0
RUN rustup default 1.72.0
RUN rustup default
# 4. Install pgrx
# I found that if pgrx version doesn't match the one in Cargo.toml, the compilation will fail
ENV PGX_HOME=/usr/lib/postgresql/${PG_VERSION_MAJOR}
RUN PGRX_VERSION=$(sed -n 's/^pgrx *=[^0-9]*\([0-9.]*\)".*/\1/p' Cargo.toml) && \
echo "PGRX_VERSION=$PGRX_VERSION" && \
cargo install cargo-pgrx --locked --force --version "${PGRX_VERSION}"
RUN rustup component add llvm-tools-preview rustc-dev
RUN rustup target install aarch64-unknown-linux-gnu
RUN rustup target install x86_64-unknown-linux-gnu
# 5. Install trusted plrust with cross-compilation
# we are in debian, so I pick `...unknown-linux-gnu`, right???
WORKDIR /tmp/plrust/plrustc
RUN ./build.sh
WORKDIR /tmp/plrust/build/bin/
RUN cp plrustc /var/lib/postgresql/.cargo/bin/
WORKDIR /tmp/plrust/plrust
RUN cargo pgrx init --pg${PG_VERSION_MAJOR} /usr/lib/postgresql/${PG_VERSION_MAJOR}/bin/pg_config
RUN PG_VER=${PG_VERSION_MAJOR} STD_TARGETS="aarch64-postgres-linux-gnu " ./build
RUN cargo pgrx install --release --features trusted --pg-config /usr/lib/postgresql/${PG_VERSION_MAJOR}/bin/pg_config
# RUN rustup component remove llvm-tools-preview rustc-dev
# RUN cargo pgrx run pg${PG_VERSION_MAJOR} --release
ENV SCRATCH_DIR="/var/lib/postgresql/plrust-scratch"
RUN mkdir -p ${SCRATCH_DIR}
WORKDIR /tmp/plrust/plrust
# RUN cat <<-EOF >> ~/.pgrx/data-16/postgresql.conf
# shared_preload_libraries = 'plrust'
# plrust.work_dir = '${SCRATCH_DIR}'
# logging_collector = on
# EOF
# RUN cat <<-EOF >> /var/lib/postgresql/data/postgresql.conf
# shared_preload_libraries = 'plrust'
# plrust.work_dir = '${SCRATCH_DIR}'
# logging_collector = 'on'
# EOF
# This works when issued manually + writting to ~/.pgrx/data-16/postgresql.conf
# RUN cargo pgrx run pg16
# 6. Reset permissions
USER root
RUN chmod 0755 `$(which pg_config) --pkglibdir` && \
cd `$(which pg_config) --pkglibdir` && \
chown root:root *.so && \
chmod 0644 *.so && \
chmod 755 `$(which pg_config) --sharedir`/extension && \
cd `$(which pg_config) --sharedir`/extension && \
chown root:root *
RUN chown root -R /usr/share/postgresql/${PG_VERSION_MAJOR}/extension/
RUN chown root -R /usr/lib/postgresql/${PG_VERSION_MAJOR}/lib/
USER postgres
# 7. Run with shared_preload_libraries=plrust + plrust.work_dir
CMD ["postgres", "-c", "shared_preload_libraries=plrust", "-c", "plrust.work_dir='/var/lib/postgresql/plrust-scratch'", "-c", "logging_collector=on"]This will give the following error:
postgres=# CREATE FUNCTION add_two_numbers(a NUMERIC, b NUMERIC) RETURNS NUMERIC STRICT LANGUAGE plrust AS $$
Ok(Some(a + b))
$$;
ERROR:
0: No such file or directory (os error 2)
Location:
plrust/src/user_crate/build.rs:126
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.I tried many many ways to build a Dockerfile and was never able to get plrust working...
Am I missing something here?
The extracted pgrx version is: 0.11.0
The rustc version is 1.72.0
The rustup default version is also 1.72.0
The Postgres major is 16