The network stack
The network stack of MoeNavigatorEngine is separated into two main entities: Network handlers and requests.
A network handler is an abstraction layer for the network facilities of the operating system. It passes data to a network connection opened by the operating system and reads data from it.
At the moment, only POSIX sockets are supported for real network connections via the NetworkHandler_POSIX class for unencrypted network data transfers and the NetworkHandler_GnuTLS class for TLS-encrypted network data transfers.
There is also a dummy network handler (NetworkHandler_Dummy) that can be used with a dummy server to test the output of requests and compare it with an expected result.
The Middleware interface allows implementations to place themselves in between a Request and a network handler to manipulate or encapsulate network traffic. Middleware implementations can be used to implement TLS support, to block requests to specific domains or to redirect traffic through a proxy.
A request is an object that sends or receives data from a network resource. In its basic form, it represents just a raw network connection.
Application layer protocols like HTTP, Gopher and others are implemented in classes derived from the basic Request class. The derived classes handle all protocol specific things like creating headers and reading out response header fields. They also make sure the header is separated from the response data.
A Request instance interacts with a network handler by telling it to open a network connection to a specified host and port. Then, the Request instance sends data to the network handler and awaits the response. To avoid polling, mutexes are used as semaphores to inicate whether data are availabe or if a response header is available.
The following request classes are available to handle application layer protocols:
- HTTPRequest: Hypertext Transfer Protocol v1.0
- GopherRequest: Gopher protocol
- QOTDRequest: Quote of the day protocol
The DataSource classes provide data in the case a request shall send data, like in HTTP POST or similar types of requests. A data source is tied to a request but the Request instance will pass the DataSource on to the NetworkHandler so that data provided by the source are directly sent out without being processed by the Request object.
The DummyDataSource is a very basic data source that outputs the exact same data that is input into it. It could be used for prototyping protocol implementations in conjunction with the Request class as the latter does not send any data by default.
The HTTPFormDataSource class has methods to set HTTP form fields and their values. These data can be encoded either using URL encoding or multipart form data. Note that the support for form fields is very basic at the moment, so that files cannot be transferred yet.
How MoeNavigatorEngine makes network requests
The following comic explains how MNE uses the components mentioned above to make a request to a network resource. In the comic, HTTPS is used, but it is the same for other protocols except that another Request and DataSource object would be used.