Although LocalCDN has included many frameworks, it is not complete and many less popular frameworks are probably not included.
In addition, manually checking each website to find out whether a framework/CDN combination is missing and then reporting it to the developer requires much work and time.
To include as many frameworks as possible, I think we can check cdnjs' repo.
That being said, there are nearly 4000 entries in cdnjs/ajax/libs/, so including all of them may require a lot of storage space.
Although LocalCDN has included many frameworks, it is not complete and many less popular frameworks are probably not included.
In addition, manually checking each website to find out whether a framework/CDN combination is missing and then reporting it to the developer requires much work and time.
To include as many frameworks as possible, I think we can check [cdnjs' repo](https://github.com/cdnjs/cdnjs/tree/master/ajax/libs).
That being said, there are **nearly 4000 entries** in `cdnjs/ajax/libs/`, so including all of them may require a lot of storage space.
So, in order for a framework mapping to be included in LocalCDN...
Find a website using XXX framework
Make sure the website calls the framework via CDN (i.e. the website does not host the framework itself)
Block the framework request (e.g. by using uBlock Origin) to see whether it is really necessary. If the website breaks, looks weird, or some features cannot be used, then it can be reported to the developer.
Like this?
So, in order for a framework mapping to be included in LocalCDN...
1. Find a website using XXX framework
2. Make sure the website calls the framework via CDN (i.e. the website does not host the framework itself)
3. Block the framework request (e.g. by using uBlock Origin) to see whether it is really necessary. If the website breaks, looks weird, or some features cannot be used, then it can be reported to the developer.
Like this?
Yes, exactly. That' s a way to test it. I'm using the developer tools of Firefox. Menu -> Web Developer -> Network. There you can block single requests.
Yes, exactly. That' s a way to test it. I'm using the developer tools of Firefox. Menu -> Web Developer -> Network. There you can block single requests.
By the way, when testing and reporting framework mappings, is it a good idea to enable the “block requests for missing resources” option?
You can do that. When I test something I use a new Firefox profile and the developer tools of Firefox. At the same time I load the extension in another profile as "Temporary Extension" and activate logging in LocalCDN.
Is that option helpful in cases like #147, when a website requests unnecessary frameworks from CDNs?
This is hard to say in general terms, which is better. I always block all non existing frameworks.
By the way, is cookieconsent eligible for inclusion?
I think so. It is not big and could be important for someone. I create a separate issue (#153)
Extract the version number of origin path jquery/2.1.4/jquery.min.js -> 2.1.4 (request-analyzer.js#L125)
Insert extracted version into the target path resources/jquery/{version}/jquery.min.jsm -> resources/jquery/2.1.4/jquery.min.jsm (request-analyzer.js#L142)
Determine and replace the target version resources/jquery/2.1.4/jquery.min.jsm -> resources/jquery/2.2.4/jquery.min.jsm (helpers.js#L427)
> By the way, when testing and reporting framework mappings, is it a good idea to enable the “block requests for missing resources” option?
You can do that. When I test something I use a new Firefox profile and the developer tools of Firefox. At the same time I load the extension in another profile as "Temporary Extension" and activate logging in LocalCDN.
> Is that option helpful in cases like #147, when a website requests unnecessary frameworks from CDNs?
This is hard to say in general terms, which is better. I always block all non existing frameworks.
> By the way, is cookieconsent eligible for inclusion?
I think so. It is not big and could be important for someone. I create a separate issue (#153)
> How does that option work?
## TL;DR:
The listener [webRequest.onBeforeRequest](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/onBeforeRequest) depends on the CDNs in [mappings.js](https://codeberg.org/nobody/LocalCDN/src/branch/main/core/mappings.js). If a CDN isn't listed there, LocalCDN ignores this request.
> Does that option block all requests to CDN domains like cdnjs.cloudflare.com, cdn.jsdelivr.net?
If this domain is listed in mappings.js, then yes.
---
## Details
Example: `cdnjs.cloudflare.com/ajax/libs/cookieconsent2/3.0.3/cookieconsent.min.js`
1. First the URL split into host and path
host = cdnjs.cloudflare.com
path = ajax/libs/cookieconsent2/3.0.3/cookieconsent.min.js
2. The supported CDNs are stored [here](https://codeberg.org/nobody/LocalCDN/src/branch/main/core/state-manager.js#L140) in the requestFilter and used [here](https://codeberg.org/nobody/LocalCDN/src/branch/main/core/state-manager.js#L148). Only matches with a CDN are processed
#### Non-existent frameworks
3. If the path part doesn't match in [mapping.js](https://codeberg.org/nobody/LocalCDN/src/branch/main/core/mappings.js#L88), it returns `false` ([here](https://codeberg.org/nobody/LocalCDN/src/branch/main/core/request-analyzer.js#L90)).
4. This `false` will be processed [here](https://codeberg.org/nobody/LocalCDN/src/branch/main/core/interceptor.js#L59)
5. LocalCDN decides [here](https://codeberg.org/nobody/LocalCDN/src/branch/main/core/interceptor.js#L79) what to do with non-existing frameworks
6. The final step for non-existent frameworks is [here](https://codeberg.org/nobody/LocalCDN/src/branch/main/core/interceptor.js#L93) with a redirection from HTTP to HTTPS if necessary.
#### Existent frameworks
Example: `cdnjs.cloudflare.com/ajax/libs/jquery/2.1.4/jquery.min.js`
3. Extract the version number of origin path `jquery/2.1.4/jquery.min.js` -> `2.1.4` ([request-analyzer.js#L125](https://codeberg.org/nobody/LocalCDN/src/branch/main/core/request-analyzer.js#L125))
4. Insert extracted version into the target path `resources/jquery/{version}/jquery.min.jsm` -> `resources/jquery/2.1.4/jquery.min.jsm` ([request-analyzer.js#L142](https://codeberg.org/nobody/LocalCDN/src/branch/main/core/request-analyzer.js#L142))
5. Determine and replace the target version `resources/jquery/2.1.4/jquery.min.jsm` -> `resources/jquery/2.2.4/jquery.min.jsm` ([helpers.js#L427](https://codeberg.org/nobody/LocalCDN/src/branch/main/modules/internal/helpers.js#L427))
Although LocalCDN has included many frameworks, it is not complete and many less popular frameworks are probably not included.
In addition, manually checking each website to find out whether a framework/CDN combination is missing and then reporting it to the developer requires much work and time.
To include as many frameworks as possible, I think we can check cdnjs' repo.
That being said, there are nearly 4000 entries in
cdnjs/ajax/libs/
, so including all of them may require a lot of storage space.The intention of LocalCDN is not to integrate and replace all existing frameworks. It's to make a website readable without contacting the CDNs.
So, in order for a framework mapping to be included in LocalCDN...
Like this?
Yes, exactly. That' s a way to test it. I'm using the developer tools of Firefox. Menu -> Web Developer -> Network. There you can block single requests.
You can do that. When I test something I use a new Firefox profile and the developer tools of Firefox. At the same time I load the extension in another profile as "Temporary Extension" and activate logging in LocalCDN.
This is hard to say in general terms, which is better. I always block all non existing frameworks.
I think so. It is not big and could be important for someone. I create a separate issue (#153)
TL;DR:
The listener webRequest.onBeforeRequest depends on the CDNs in mappings.js. If a CDN isn't listed there, LocalCDN ignores this request.
If this domain is listed in mappings.js, then yes.
Details
Example:
cdnjs.cloudflare.com/ajax/libs/cookieconsent2/3.0.3/cookieconsent.min.js
First the URL split into host and path
host = cdnjs.cloudflare.com
path = ajax/libs/cookieconsent2/3.0.3/cookieconsent.min.js
The supported CDNs are stored here in the requestFilter and used here. Only matches with a CDN are processed
Non-existent frameworks
If the path part doesn't match in mapping.js, it returns
false
(here).This
false
will be processed hereLocalCDN decides here what to do with non-existing frameworks
The final step for non-existent frameworks is here with a redirection from HTTP to HTTPS if necessary.
Existent frameworks
Example:
cdnjs.cloudflare.com/ajax/libs/jquery/2.1.4/jquery.min.js
Extract the version number of origin path
jquery/2.1.4/jquery.min.js
->2.1.4
(request-analyzer.js#L125)Insert extracted version into the target path
resources/jquery/{version}/jquery.min.jsm
->resources/jquery/2.1.4/jquery.min.jsm
(request-analyzer.js#L142)Determine and replace the target version
resources/jquery/2.1.4/jquery.min.jsm
->resources/jquery/2.2.4/jquery.min.jsm
(helpers.js#L427)Many thanks for the detailed explanation!
You're welcome 😊