Pi-hole As A Snapcraft Package

Jacob Salmela recently went to the Snapcraft Summit in London and worked with Alan Pope for a few days to attempt to make Pi-hole installable via snap.  They made some progress, but it’s not quite functional enough to publish in the Snap Store.  To that end, we wanted to make the repo public to see if anyone in the community had anything to contribute.  Feel free to take a look, open issues, or make pull requests.

https://github.com/pi-hole/pihole-snap

This was also a historic event because Jacob Salmela and Adam Warner (PromoFaux) met up in real life.  Our development team is 100% remote and we’ve worked together for years, but no one has ever physically met one another.  Here’s a picture of them in front of The Shard.

Announcing Our RESTful API — Contributions Welcome

In addition to our next gen Web interface, we have been building a new RESTful API.  Until today, this repo has been private, but it is now ready for community feedback and help in maturing it for production.

View the API on GitHub

One reason for making this public now is that we believe that the API is ready for you to look at and experiment with. The API is at the same point in its development as FTL was when we first shared that project with the world, and it benefited from the feedback it got in those early stages.

Another reason for opening up the project now is that many of us are back in school or our jobs have been busier. We’re hoping by make the work in progress code available now, some community members would help contribute to it and provide feedback.

Goals for the API

As part of replacing the current PHP web interface, the work that was done by the backend PHP code must be moved to a new subsystem. The new web interface only runs in the browser (unlike PHP), so it can not perform the backend duties that the PHP code performed (such as enabling/disabling blocking). To perform these backend jobs, the new API was created.

The API handles the server-side work necessary to support the web interface. This includes communicating with FTL and retrieving the stats it generates, as well as managing the white/black/regex lists and settings. PHP managed to do these tasks, but often issues arose due to the dynamic nature of PHP and the organic way in which the codebase evolved. The new API needed to be able to handle these tasks without introducing bugs and while providing an extendable codebase which could be easily tested.

In addition to PHP, lighttpd is often mentioned in issues, including how it recently broke the config option we use to provide an optional external config. Because the new Web interface only requires a static file server, the API will host the Web interface. This will give us more control over the web server, make changing the Web interface’s port simple, and make Pi-hole easier to package into a Docker container.

Technical Details

The API is written in Rust, a language new to the Pi-hole project. Rust is a safe and fast language which matches well with our goals for the API. It is statically typed and prevents whole categories of errors while being productive and extendable.

The API exposes a RESTful HTTP API which is used by the web interface, but also is general enough for other applications. We have seen some users interested in using Pi-hole’s (PHP) API, which often required them to read the API’s code before understanding how to use it. With this new API, we are making that experience better by following RESTful practices and providing a more intuitive interface.

The API (and the new web interface) were started with testing in mind, and both boast a suite of fine-grained tests. As Pi-hole organically grew, we sometimes ran into issues (in production), which could have been easily found by testing. There are some small test suites in some subsystems, but the system which often has the most bugs (the PHP web interface and API) is very hard to test and must rely on manual testing. This does not scale, and so we strove to find a system which would make testing both a breeze and would be less likely to break. This led us to Rust and the architecture we use in the API’s codebase.

The main way the API communicates with FTL is via shared memory. This is still a work in progress, but it is proving to be very effective (API branch, FTL branch). The idea is that FTL will store its data in a shared-memory space so that the API can come in and read from that memory to fulfill requests. This makes their communication very fast and easy to code. Also, because they share memory instead of sending messages over sockets, the communication is statically typed. This means that instead of assuming the type of the data coming from FTL, the API knows exactly what type the data is in. The PHP API had issues which arose out of how it serialized the data returned by FTL. It had to read the data as a string and assume how it was structured. This shared memory connection will either eliminate or greatly reduce those issues.

This shared memory system means FTLDNS will be even lighter as it no longer needs to do the work of preparing the data for the API (as it did for PHP). Instead, it will concentrate on generating the statistics, and the API can read FTL’s data directly.

So take a look at the code for our new API and feel free to send us a pull request.

https://github.com/pi-hole/api

Thanks again for all of your continued support.

What Really Happens On Your Network? Part Eight

Just last week, we had a post of things people have discovered happening on their networks.  But there is no shortage of these types of posts, so here is another collection of them or you can read previous iterations of these type of posts.

Read on to find out more to find out what people discovered happening on their networks, thanks to Pi-hole. Continue reading “What Really Happens On Your Network? Part Eight”

What Really Happens On Your Network? Part Seven

We’re back with the latest iteration of users discovering things on their network via Pi-hole.  This post is a compilation of things users have discovered over the past year.  Some were bad, some were interesting, and some were enlightening.  This isn’t the first time we’ve written a post like this, but we will try to go into more detail about what people have discovered and group together similar discoveries.  Below you’ll find previous renditions of this type of post.

Read on to find out more to find out what people discovered happening on their networks, thanks to Pi-hole.

Continue reading “What Really Happens On Your Network? Part Seven”

Mitigate A New CERT Vulnerability (#598349) With An Entry In /etc/hosts

There is a new CERT vulnerability that can leave you vulnerable to a Man-in-the-Middle attack.  You can mitigate this vulnerability today by adding these two lines to your /etc/hosts file:

0.0.0.0 wpad wpad.example.com 
:: wpad wpad.example.com

example.com is a stand in for your local domain.  So replace example.com with whatever your local domain is.

The essence of this vulnerability is that an attacker can add a device to the network named wpad and get a DHCP lease, thus inserting the name wpad.example.com in the local DNS pointing to the attacker’s machine.  The presence of that A record allows control of the proxy settings of any browser in the network.

You can learn more about the technology behind this attack at Google’s Project Zero page–it’s an older article, but gives some insight into the inner workings of the attack.

The next release of dnsmasq includes an option (dhcp-ignore-names) that can be used to mitigate the attack at the source, but we haven’t heard how Simon will act on this new vulnerability.

Since FTLDNS is just our fork of dnsmasq, we can easily merge in any upstream changes from him, but we wanted to let you know of the /etc/hosts fix that you can immediately implement.

Pi-hole v4.0 Released With FTLDNS, Improved Blocking Modes, Regex, Docker, and More

We’re very pleased to release Pi-hole v4.0 today, which includes fixes, tweaks, and lots of new stuff, including FTLDNS (special thanks to our beta testers!)  In a sentence, FTLDNS is dnsmasq with Pi-hole’s special sauce baked in.

FTLDNS does everything dnsmasq does because it is dnsmasq–just our fork of it.  So all of your existing config files will still work with it.  We intentionally modified the original dnsmasq source code as little as possible so that we can easily integrate any upstream changes as they are released.

Read on to find out everything included in this release or read the technical details in the changelogs.

Continue reading “Pi-hole v4.0 Released With FTLDNS, Improved Blocking Modes, Regex, Docker, and More”

FTLDNS and Unbound Combined For Your Own All-Around DNS Solution

How Pi-hole Works

Pi-hole acts as a forwarding DNS server, which means if it doesn’t know where a domain is, it has to forward your query to another server that does.  When you install Pi-hole, it knows where the ad-serving domains are (because you tell it), so it doesn’t forward those requests.  But it doesn’t know where legitimate sites are. Thus these requests are forwarded to an upstream, recursive server.

These servers also don’t know where the real Website exists unless they have been asked to find it before.  The only DNS servers that truly know where a domain is is an authoritative DNS server.  For now, we don’t need to know what an authoritative DNS server is, just that it’s the single source of truth for a domain’s real IP address.

So when you have a Pi-hole in use on your network, the flow of traffic goes like this:

  1. Your client asks the Pi-hole Who is pi-hole.net?
  2. Your Pi-hole will check its cache and reply if the answer is already known.
  3. Your Pi-hole will check the blocking lists and reply if the domain is blocked.
  4. Since neither 2. nor 3. is true in our example, the Pi-hole forwards the request to the configured external upstream DNS server(s).
  5. Upon receiving the answer, your Pi-hole will reply to your client and tell it the answer of its request.
  6. Lastly, your Pi-hole will save the answer in its cache to be able to respond faster if any of your clients queries the same domain again.

The Concern With Upstream Servers

The concern with the existing method lies in step 4.  In today’s world, these upstream servers are known as Google, OpenDNS, and CloudFlare, amongst others.  They advertise themselves as free private DNS servers, but how do you know for certain they are keeping their promise that your information is truly private?

Furthermore, from the point of an attacker, the DNS servers of larger providers are very worthwhile targets, as they only need to poison one DNS server, but millions of users might be affected.  For example, instead of your bank’s actual IP address, you could be sent to a phishing site hosted on some island.  This scenario has already happened and it isn’t unlikely to happen again…

So What Is The Difference Between A Recursive DNS Server and An Authoritative DNS server?

The first distinction we have to be aware of is whether a DNS server is authoritative or not.  If I’m the authoritative server for, e.g., pi-hole.net, then I know which IP is the correct answer for a query.  Recursive name servers, in contrast, resolve any query they receive by consulting the servers authoritative for this query by traversing the domain.  Example: We want to resolve pi-hole.net. On behalf of the client, the recursive DNS server will traverse the path of the domain across the Internet to deliver the answer to the question.

What Is The Solution?

Operating your own local, recursive DNS server.  Think of it as running your own Google or CloudFlare DNS service.  It can run on the same device you are already using Pi-hole for and there are no additional hardware requirements.

This changes the six step procedure mentioned previously to this 12 step process:

How Pi-hole Works With FTLDNS and Unbound

  1. Your client asks the Pi-hole Who is pi-hole.net?
  2. Your Pi-hole will check its cache and reply if the answer is already known.
  3. Your Pi-hole will check the blocking lists and reply if the domain is blocked.
  4. Since neither 2. nor 3. is true in our example, the Pi-hole delegates the request to the (local) recursive DNS resolver.
  5. Your recursive server will send a query to the DNS root servers: “Who is handling .net?”
  6. The root server answers with a referral to the TLD servers for .net.
  7. Your recursive server will send a query to one of the TLD DNS servers for .net: “Who is handling pi-hole.net?”
  8. The TLD server answers with a referral to the authoritative name servers for pi-hole.net.
  9. Your recursive server will send a query to the authoritative name servers: “What is the IP of pi-hole.net?”
  10. The authorative server will answer with the IP address of the domain pi-hole.net.
  11. Your recursive server will send the reply to your Pi-hole which will, in turn, reply to your client and tell it the answer of its request.
  12. Lastly, your Pi-hole will save the answer in its cache to be able to respond faster if any of your clients queries the same domain again.

Step 4 is where the major change happens.  The steps that follow are what the upstream servers would normally handle (along with any data tracking they may or may not be doing).

Recursion is more involved than just asking some upstream server. This has benefits and drawbacks:

Benefit

Privacy – as you’re directly contacting the responsive servers, no server can fully log the exact paths you’re taking, as e.g. the Google DNS servers will only be asked if you want to visit a Google website, but not if you visit the website of your favourite newspaper, etc.

Drawback

Traversing the entire path may be slow, especially the first time you visit a website.  While the bigger DNS providers always have answers for commonly used domains in their cache, you will have to transverse the path if you visit a page for the first time time.  A first request to a formerly unknown TLD may take up to one second (or even more if you’re also using DNSSEC).  Subsequent requests to domains under the same TLD usually complete in < 0.1s. Fortunately, your Pi-hole does efficient caching to minimize the number of queries that will actually have to be performed.

Setting It Up

This blog post won’t go into detail about how to set it up, for that, we have an article you can follow (at this time you’ll need to be participating in our beta test of FTLDNS).

But once complete, you’ll be able to do something like this instead of setting Google as your upstream DNS provider.

Interested in this?

FTLDNS is currently in beta, so we hope you’ll check it out and also help support our endeavors with our one-time fundraising goal of $100,000.

FTLDNS is out of beta and part of Pi-hole v4.0.

 

Limited Edition Pi-hole Coins Are Now For Sale

Patrons got first dibs, but there are still plenty of coins available.

These coins are high quality, colored, and textured.  Check out the product page for more information.  Enjoy!

Proceeds from the sales will be used to further develop Pi-hole.