Pi-hole FTL v5.9, Web v5.6 and Core v5.4 released

Pi-hole FTL v5.9, Web v5.6 and Core v5.4 released

As always, please read through the changelog before updating with pihole -up.

After a successful beta round, we are excited to announce that the next version(s) of Pi-hole are now available in the main branches! A great big “Thank You” to all those who not only tested the beta, but actively reported back with any issues they found, so that we could fix them before release.

As a reminder, if you have been testing the beta – to switch back to this release simply run pihole checkout master

And before anyone asks… New tag for docker image will arrive shortly, once it’s been tested!

Highlights

  • Update embedded dnsmasq DNS server to version 2.86
    • Handle DHCPREBIND requests in the DHCPv6 server code
    • Fix bug which caused dnsmasq to lose track of processes forked to handle TCP DNS connections under heavy load
      The code checked that at least one free process table slot was available before listening on TCP sockets, but didn’t take into account that more than one TCP connection could arrive, so that check was not sufficient to ensure that there would be slots for all new processes. It compounded this error by silently failing to store the process when it did run out of slots. Even when this bug is triggered, all the right things happen, and answers are still returned. Only under very exceptional circumstances, does the bug manifest itself: see https://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2021q2/014976.html 
    • Major rewrite of the DNS server and domain handling code
      This should be largely transparent, but it drastically improves performance and reduces memory foot-print when configuring large numbers domains of the form local=/adserver.com/ or local=/adserver.com/# Lookup times now grow as log-to-base-2 of the number of domains, rather than greater than linearly, as before. The change makes multiple addresses associated with a domain work address=/example.com/1.2.3.4 address=/example.com/5.6.7.8
      It also handles multiple upstream servers for a domain better; using the same try/retry algorithms as non domain-specific servers. This also applies to DNSSEC-generated queries. Finally, some of the oldest and gnarliest code in dnsmasq has had a significant clean-up. It’s far from perfect, but it is better.
    • Revise resource handling for number of concurrent DNS queries
      This used to have a global limit, but that has a problem when using different servers for different upstream domains. Queries which are routed by domain to an upstream server which is not responding will build up and trigger the limit, which breaks DNS service for all other domains which could be handled by other servers. The change is to make the limit per server-group, where a server group is the set of servers configured for a particular domain. In the common case, where only default servers are declared, there is no effective change.
    • Improve efficiency of DNSSEC
      The sharing point for DNSSEC RR data used to be when it entered the cache, having been validated. After that queries requiring the KEY or DS records would share the cached values. There is a common case in dual-stack hosts that queries for A and AAAA records for the same domain are made simultaneously. If required keys were not in the cache, this would result in two requests being sent upstream for the same key data (and all the subsequent chain-of-trust queries.) Now we combine these requests and elide the duplicates, resulting in fewer queries upstream and better performance. To keep a better handle on what’s going on, the extra logging mode has been modified to associate queries and answers for DNSSEC queries in the same way as ordinary queries. The requesting address and port have been removed from DNSSEC logging lines, since this is no longer strictly defined.
    • Allow wildcards in dnsmasq config domain patterns
      Domain patterns in --address, --server and --local have, for many years, matched complete labels only, so --server=/google.com/1.2.3.4 will apply to google.com and www.google.com but NOT supergoogle.com. dnsmasq now introduces an optional * at the LHS of the domain string which changes this behaviour so as to include substring matches within labels. So, --server=/*google.com/1.2.3.4 applies to google.com and www.google.com AND supergoogle.com
    • FTL also imported the requested feature to Support Cisco Umbrella/OpenDNS Device ID Remote IP FTL#1096 
    • Connection track mark based DNS query filtering
    • Allow smaller than 64 prefix lengths in synth-domain, with caveats. --synth-domain=1234:4567::/56,example.com is now valid.
    • Make domains generated by --synth-domain appear in replies when in authoritative mode.
    • Ensure CAP_NET_ADMIN capability is available when conntrack is configured.
    • When --dhcp-hostsfile --dhcp-optsfile and --addn-hosts are
      given a directory as argument, define the order in which files within that directory are read (alphabetical order of filename).
  • Interface-dependent handling of pi.hole and the machine’s hostname
    This makes FTL automatically reply with the appropriate IP address to both pi.hole and the machines hostname. Before this change, FTL always used a hard-coded address set during the weekly gravity updates (pihole -g). The new method is interface-aware and may reply with different addresses on different interfaces (e.g. Ethernet, WiFi or Wireguard network). The address FTL replies with can be overwritten using the REPLY_ADDR4/6 settings in /etc/pihole/pihole-FTL.conf.
  • Show automatically generated DNSSEC queries
    After each restart of pihole-FTL, there will be a high number of DNSSEC-related queries (DNSKEY and DS) as the DNSSEC chain of trust needs to build for all domains queried in your network. The number of queries will quickly drop afterwards when the chain has been primed with all the top-level domains you are typically visiting. You should then only rarely see a DS query when visiting an altogether new webpage. To ensure this new information is valuable for you, we will sketch how DNSSEC validation works in another upcoming blog post. If you dont want to see these queries, you can easily set SHOW_DNSSEC=false in /etc/pihole/pihole-FTL.conf to suppress analyzing them altogether (they will still be happening in the background).
  • Update embedded SQLite engine to version 3.36
    1. Improvement to the EXPLAIN QUERY PLAN output to make it easier to understand.
    2. Byte-order marks at the start of a token are skipped as if they were whitespace.
    3. An error is raised on any attempt to access the rowid of a VIEW or subquery. Formerly, the rowid of a VIEW would be indeterminate and often would be NULL.
    4. The memdb VFS now allows the same in-memory database to be shared among multiple database connections in the same process as long as the database name begins with /.
    5. Back out the EXISTS-to-IN optimization (item 8b in the SQLite 3.35.0 change log 1) as it was found to slow down queries more often than speed them up.
    6. Improve the constant-propagation optimization 1 so that it works on non-join queries.
    7. The REGEXP extension 2 is now included in CLI builds (use pihole-FTL sqlite in your terminal to access the embedded SQLite engine).
  • Enable .recover option for embedded SQLite engine
    Exemplary use to repair a corrupted long-term database:
    pihole-FTL /etc/pihole/pihole-FTL.db ".recover" | pihole-FTL ~/pihole-FTL_recovered.db
    On success, the old database can be replaced by the repaired one.
  • Fix dnsmasq --server option interpretation (fix sent and accepted upstream)
  • Allow users to configure how FTL reacts to queries when the gravity database is not available (REPLY_WHEN_BUSY option )
  • Rate-limiting events are shown in the Pi-hole diagnosis system so you get notified on the web dashboard if any of your clients is rate-limited
  • Also display DNSSEC status of cache replies and internally generated DNSSEC-related queries
  • Add new regex extension ;reply=... to force specific replies on regex match
  • Other changes and bugfixes
    • Improve reliability under heavy TCP query load
    • Log when listening on the wildcard address. This will help debugging edge-case setups.
    • Fix crash when bind-address is used.
    • Fix for incorrect (but harmless) FATAL: Trying to access query ID -1 errors messages in pihole-FTL.log
    • Show hostnames also for clients specified by MAC address
    • Improve warning messages for defect hwclocks
    • Delay startup of FTL earlier to avoid database importing issues with incorrectly configured fake-clocks (if applicable)
    • Add a new theme that can automatically switch to dark mode if used on the device
    • Enable PHP8 support for Debian/Ubuntu
    • Be explicit in IPv6 RA values
    • Don’t overwrite existing logrotate script
    • Less coloring in the Query Log to enhance clarity

 

Full Release Notes:

Core:

  • Enable PHP8 support for Debian/Ubuntu #4135
  • Cosmetic change to pihole -v -c output #4143
  • Re-remove wget from dependencies #4153
  • Added docker to README #4175
  • Be explicit in IPv6 RA values. #4206
  • Don’t overwrite existing logrotate #4207
  • Fix typo #4199
  • Add new tests to ensure dependencies are available to the OS #4215
  • Filter pihole -t #4191
  • Add missing sed anchors when deleting lines. #4178
  • Remove false statement about dependency removal on Raspbian #4158
  • Add tail of pihole.log to debug output #4162
  • Debug Log bugfix #4216
  • Handle pi.hole and hostname in FTL instead of local.list #4131
  • Remove mod_compress (mod_deflate) from lighttpd to provide backwards compatible support for Debian Bullseye (11) #4221
  • Remove mod_compress from lighttpd config on fedora/centos #4222
  • Support and test Debian Bullsye (11) and Ubuntu 21.04 #4126
  • Delete supportedos.txt – it is not actually used anywhere can can cause confusion #4228
  • Fix coloring of pihole -t #4230
  • Add -t (teleporter) option to Admin Console help dialog #4233
  • Guard for logrotate func non-zero return #4242
  • Make output of SHM dir human readable #4244
  • Remove extra failure display when installation of logrotate file is skipped because the file already exists #4247
  • Fix error on checking interfaces that are not dual-stack #4248
  • Increase width of ID column in adlist and domain table in debug script #4262
  • Give pihole its own logrotate state file #4225
  • Remove comparison of IP addresses with setupVars.conf #4246
  • Some tweaks to how packages are installed, plus removal of dhcpd5 dependency (Option 1) #4260
  • Add uptime to debug log #4265
  • Add switching ‘to…from’ message to ftl checkout output #4266
  • On enabling/disabeling only reload-lists instead of reload #4264
  • Fix Splashpage not appearing properly on non-root directories #4271
  • Conditional forwarding: Also forward unqualified host names #4287
  • Add sudo to apt-get recommendation #4292
  • Fix pipe to pihole tricorder #4304
  • Add content of /etc/resolv.conf to debug output #4311
  • Use –no-progress-meter instead of –silent so error reasons are printed #4305
  • Forward unqualified names to the CF target only when the “Never forward non-FQDN” option is NOT ticked #4317
  • Remove ineffective Access-Control-Allow-Origin header #4275
  • Add content of /etc/dnsmasq.conf to debug output #4312
  • Add possibly missing capabilities #4314
  • Install RFC6761 config file #4282
  • Include all files in the dnsmasq config dir in the debug output #4243

Web

  • Password autocompletion #1785
  • Try to obtain hostname for MAC clients #1793
  • Improvement to readability of footer “update” command text #1797
  • Added the option for an automatic dark mode based on the device status #1836
  • Add reply type DNSSEC #1837
  • Adding CORS support via environment variable #1822
  • Fix domain validation method #1852
  • Fix require func.php #1855
  • Add interpretation for Pi-hole message type RATE_LIMIT #1859
  • Stop timer when user reenabled blocking early #1863
  • Fix TypeError if no extended DNS error is available #1862
  • Import two fixes #1867
  • Add new blocked by database status and NONE reply type #1869
  • Add httponly = true to persistent login cookie #1875
  • Apply htmlentities in a couple of places to prevent xss #1876
  • Clarify how and when UQDN are forwarded with conditional forwarding #1873
  • Less color for the Query Log #1872
  • Add BLOB reply type #1871
  • footer.php: move FTL before Web Interface #1854
  • Remove adlists tab from settings page #1845

FTL

  • Customizable locking while database is busy #1156
  • Regex extension: Specify reply type #1159
  • Update embedded dnsmasq to version 2.86
  • Improve locking during heavy TCP forking #1134
  • Log when listening on the wildcard address #1135
  • Improve warning messages for defect hwclocks #1136
  • Fix crash when bind-address is used #1132
  • And everything else at the top of this blog post

Comments are closed.

©  2024 Pi-hole.