Pyfisch’s Website > Blog

HTTP Crate with URL Support & a Simple HTTP Client

written by Pyfisch on

While the http crate generally has a great API I have been unsatisfied how it handles URLs. To create a HTTP request a full URL is needed with a scheme (http/https), authority ( and a path (/search?q=rust) but http does enforce this and allows you to only state the path. This means both clients and servers are either unable to determine protocol and and authority information or have to do this manually.

There is already the well-established rust-url crate but it is not used by http. This has been criticized by multiple people as the http crate does not even provide conversion functions for better ergonomics. As Armin Ronacher (mitsuhiko) puts it:

With the url crate we have a pretty damn good (new) standard compliant crate that is very popular. Right now I use this everywhere and then when doing requests to hyper I need to convert them which is not very great ergonomics and performance wise.

I’ve forked the http crate and created my own http-with-url demo. Each request is created with a complete URL:

let url = Url::parse("");
let request = Request::new(url, ());

A client can deconstruct the URL into pieces: in HTTP/1.1 is the Host header, /search?q=rust is part of the request line and https tells the client to establish a secure connection. A server already knows the protocol and can build from the Host header and the path a complete URL. Currently this task is left to the application authors by the http crate.

Some edge cases:

HTTP demo client

To demonstrate how to use the http-with-url crate I have written a simple client.

extern crate boguin;
extern crate http_with_url as http;

fn main() {
    let mut client = boguin::Client::new();
    let url = http::Url::parse("").unwrap();
    let request = http::Request::new(url, ());
    let response: http::Response<String> = client.fetch(request).expect("request works");
    println!("{}", response.status());
    println!("{}", response.body());

It combines a few existing crates to parse HTTP (httparse), provide proven TLS encryption (native-tls) and of course the http crate itself. With these crates I found I quite easy to write the “glue” code to build a functioning HTTP client.

Also try out the command line client.