||1 month ago|
|.github||4 months ago|
|libias@a175135398||1 month ago|
|overlay/Mk||6 months ago|
|vendor||1 month ago|
|.cirrus.yml||1 month ago|
|.gitignore||6 months ago|
|.gitmodules||1 month ago|
|.kakrc.local||6 months ago|
|CHANGELOG.adoc||1 month ago|
|LICENSE||6 months ago|
|Makefile||1 month ago|
|README.adoc||6 months ago|
|build.ninja||6 months ago|
|build.ninja.spec||1 month ago|
|configure||6 months ago|
|loop.c||6 months ago|
|loop.h||6 months ago|
|parfetch.c||3 months ago|
|progress.c||6 months ago|
|progress.h||6 months ago|
Fetch FreeBSD ports with parallel connection support and connection pipelining.
|This is an experiment. Use at your own risk.|
This is a glue application between libcurl and the ports
framework. libcurl does all the heavy lifting. It can be
activated by enabling an overlay. It replaces the default
A statically linked binary is available in the overlay to make it easy to use with Poudriere(-devel) as well.
The ports framework does not make use of modern features like
HTTP pipelining. All
USES=cargo ports fetch most distfiles from
a single host (https://crates.io) so could benefit greatly from
it. Ports with hundreds of distfiles call
fetch(1) hundreds of
times and will open hundreds of connections one after the other.
As an example here is a basic time comparison for
where instead of opening >150 connections to https://crates.io
sequentially, it just opens 1 connection to it and can fetch all
crates basically immediately.
Even ports like
x11-toolkits/wlroots with only a handful of
distfiles can benefit from it.
$ time make -C devel/tokei distclean checksum 43.86 real 3.52 user 5.04 sys $ time make -C x11-toolkits/wlroots distclean checksum 12.28 real 1.25 user 1.81 sys
$ time make -C devel/tokei OVERLAYS=/usr/local/share/parfetch/overlay distclean checksum 3.24 real 1.24 user 1.22 sys $ time make -C x11-toolkits/wlroots OVERLAYS=/usr/local/share/parfetch/overlay distclean checksum 2.79 real 0.85 user 0.85 sys
$ ./configure && ninja
$ ninja install
Local ports setup
Enable the overlay in
This requires a Poudriere version with overlay support.
Make the overlay available to Poudriere:
$ poudriere ports -c -p parfetch -m null -M /usr/local/share/parfetch/overlay
devel/tokei and use Parfetch to fetch distfiles:
$ poudriere bulk -O parfetch devel/tokei
Options can be set in
When defined during makesum, distinfo is created/updated but no distfiles are saved to disk. Note that the files are still downloaded completely to checksum them but DISTDIR is left untouched.
When defined during makesum, retain the previous TIMESTAMP in distinfo. This can be useful when refreshing patches that have no code changes and thus do not warrant a TIMESTAMP bump.
This sets the maximum number of simultaneous open connections to a single host.
Default is 1.
This sets the global connection limit. Parfetch will not use more than this number of connections.
Default is 4.