BSD Now

1 年前
-
-
(基於 PinQueue 指標)
BSD Now
Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros. The show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day.
Thu, 16 Feb 2023 08:45:06 -0000
494: Unix workstation extinction
Mass extinction of UNIX workstations, Determine Who Can Log In to an SSH Server, Factors When Considering FreeBSD vs. Linux Packages, A Visual Guide to SSH Tunnels, Harvesting the Noise While it’s Fresh, Bastille - The Jail Manager on FreeBSD, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines The mass extinction of UNIX workstations (https://www.osnews.com/story/135605/the-mass-extinction-of-unix-workstations/) whoarethey: Determine Who Can Log In to an SSH Server (https://www.agwa.name/blog/post/whoarethey) News Roundup FreeBSD vs. Linux 5 Factors When Considering FreeBSD vs. Linux: Packages (https://klarasystems.com/articles/freebsd-vs-linux-5-factors-when-considering-freebsd-vs-linux-package-management/) A Visual Guide to SSH Tunnels: Local and Remote Port Forwarding (https://iximiuz.com/en/posts/ssh-tunnels/) Harvesting the Noise While it’s Fresh, Revisited (https://medium.com/@peter.hansteen/harvesting-the-noise-while-its-fresh-revisited-3da1894cc8a7) Bastille - The Jail Manager on FreeBSD (https://byte--sized-de.translate.goog/linux-unix/bastille-der-jail-manager-unter-freebsd/?_x_tr_sl=de&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=wapp) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 16 Feb 2023 00:00:00 -0800
493: Dotfile Management
Write Admin tools from Day One, Differentiating between Data Security and Data Integrity, 45 year-old Unix tool is finally getting an upgrade, OpenBSD 7.2 on an ODROID-HC4, Dotfiles Management, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Write Admin tools from Day One (https://milwaukeemaven.blogspot.com/2022/08/write-admin-tools-from-day-one.html) Differentiating between Data Security and Data Integrity (https://klarasystems.com/articles/openzfs-data-security-vs-integrity/) News Roundup This 45 year-old Unix tool is finally getting an upgrade (https://www.techradar.com/news/45-year-old-unix-tool-finally-gets-an-upgrade) Installing OpenBSD 7.2 on an ODROID-HC4 (https://www.tumfatig.net/2022/install-openbsd-odroid-hc4/) Dotfiles Management (https://mitxela.com/projects/dotfiles_management) Beastie Bits FreeBSD Journal - November/December 2022 - Observability and Metrics (https://freebsdfoundation.org/past-issues/observability-and-metrics/) HAMMER2 file system for NetBSD (https://github.com/kusumi/netbsd_hammer2) Running OpenBSD 7.2 on your laptop is really hard (not) (https://sohcahtoa.org.uk/openbsd.html) MinIO on OpenBSD 7.2: Install (https://dev.to/nabbisen/minio-on-openbsd-72-install-3b3h) WireGuard VPN on OpenBSD (https://www.adrianobarbosa.xyz/blog/openbsd-wireguard.html) A tool for glamorous shell scripts (https://github.com/charmbracelet/gum) Visualize your git commits with a heat map in the terminal (https://github.com/james-stoup/heatwave) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 09 Feb 2023 00:00:00 -0800
492: Feeling for NetBSD
Writing your own operating system, Continuous Integration and Quality Assurance Update, feeling for the NetBSD community, Testing wanted: execute-only on amd64, GCC uses Modula-2 and Rust, do they work on OpenBSD, Unix is dead; long live Unix, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Part 1: Writing your own operating system (https://o-oconnell.github.io/2023/01/12/p1os.html) 2022 in Review: Continuous Integration and Quality Assurance Update (https://freebsdfoundation.org/blog/2022-in-review-continuous-integration-and-quality-assurance-update/) News Roundup I feel for the NetBSD community (https://rubenerd.com/i-feel-for-the-netbsd-community/) Testing wanted: execute-only on amd64 (https://www.undeadly.org/cgi?action=article;sid=20230115095258) GCC now includes Modula-2 and Rust. Do they work on OpenBSD? (https://briancallahan.net/blog/) Unix is dead. Long live Unix! (https://www.theregister.com/2023/01/17/unix_is_dead/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions • [Kevin - Advent of Computing podcast covers BSD](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/492/feedback/Kevin%20-%20Advent%20of%20Computing%20podcast%20covers%20BSD.md) • [ilo - thanks](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/492/feedback/ilo%20-%20thanks.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 02 Feb 2023 00:00:00 -0800
491: Catch the Spammers
Dragonfly BSD 6.4 is out, Running OpenZFS – Choosing Between FreeBSD and Linux, OpenBSD Mastery: Filesystems ebook leaks, catching 71% spam, crazy unix shell prompts, Linux Binary Compatibility: Ubuntu on FreeBSD, Reproducible Builds Summit Venice 2022, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Dragonfly BSD 6.4 is out (https://www.dragonflybsd.org/release64/) Running OpenZFS – Choosing Between FreeBSD and Linux (https://klarasystems.com/articles/running-openzfs-choosing-between-freebsd-and-linux/) News Roundup “OpenBSD Mastery: Filesystems” ebook leaking out (https://mwl.io/archives/22462) Can Your Spam-eater Manage to Catch Seventy-one Percent Like This Other Service? (https://bsdly.blogspot.com/2022/12/can-your-spam-eater-manage-to-catch.html) Crazy unix shell prompts (https://lists.nycbug.org:8443/pipermail/semibug/2022-December/000775.html) Linux Binary Compatibility: Ubuntu on FreeBSD (https://byte--sized-de.translate.goog/linux-unix/linux-binary-compatibility-ubuntu-unter-freebsd/?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=wapp) Reproducible Builds Summit Venice 2022 (https://blog.netbsd.org/tnf/entry/reproducible_builds_summit_venice_2022) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Felix - Managing Jails with ansible (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/491/feedback/Felix%20-%20Managing%20Jails%20with%20ansible.md) John Baldwin - bhyve networking setup article (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/491/feedback/John%20Baldwin%20-%20bhyve%20networking%20setup%20article.md) Welton - bhyve webadmin (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/491/feedback/Welton%20-%20bhyve%20webadmin.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 26 Jan 2023 00:00:00 -0800
490: New Year’s Plan9’ing
FreeBSD Foundation’s Software Development review of 2022, what can we learn from Vintage Computing, OpenBSD KDE Status Report 2022, a Decade of HardenedBSD, In Praise of Plan9, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines 2022 in Review: Software Development (https://freebsdfoundation.org/blog/2022-in-review-software-development/) What can we learn from Vintage Computing (https://github.com/readme/featured/vintage-computing) News Roundup OpenBSD KDE Status Report 2022 (https://www.sizeofvoid.org/posts/2022-26-12-openbsd-kde-status-report-2022/) A Decade of HardenedBSD (https://git.hardenedbsd.org/shawn.webb/articles/-/blob/master/hardenedbsd/2023-01_decade/article.md) In Praise of Plan9 (https://drewdevault.com/2022/11/12/In-praise-of-Plan-9.html) Beastie Bits LibreSSL 3.7.0 Released (https://undeadly.org/cgi?action=article;sid=20221212183516) OPNsense 22.7.10 released (https://opnsense.org/opnsense-22-7-10-released/) BSDCan 2023 call for papers (https://lists.bsdcan.org/pipermail/bsdcan-announce/2022-December/000194.html) How to lock OpenSSH authentication agent (https://sleeplessbeastie.eu/2022/12/28/how-to-lock-openssh-authentication-agent/) Once upon a time long ago, I was sitting alone in the UCLA ARPANET site... (https://mastodon.laurenweinstein.org/@lauren/109588605178700335) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 19 Jan 2023 00:00:00 -0800
489: Refreshing Perspective
FreeBSD vs. Linux – Networking, HDMI sound output through TV speakers on FreeBSD 13, Getting started with tmux, Samba Active Directory, OpenIKED 7.2 released, FreeBSD Plasma 5 GUI Install, DHCP server howto in German, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD vs. Linux – Networking (https://klarasystems.com/articles/freebsd-vs-linux-networking/) (Solved), HDMI sound output through TV speakers Freebsd 13 or @4 plus VCHIQ audio patch - Raspberry Pi Forums (https://forums.raspberrypi.com/viewtopic.php?t=343233) News Roundup Getting started with tmux (https://ittavern.com/getting-started-with-tmux/) Samba Active Directory (https://cromwell-intl.com/open-source/samba-active-directory/freebsd-raspberry-pi.html) OpenIKED 7.2 released (http://undeadly.org/cgi?action=article;sid=20221202230711) FreeBSD Plasma 5 GUI Install (https://byte--sized-de.translate.goog/linux-unix/freebsd-kde-plasma-5-als-gui-installieren/?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en-US&_x_tr_pto=wapp) Original German Article (https://byte-sized.de/linux-unix/freebsd-kde-plasma-5-als-gui-installieren/) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 12 Jan 2023 00:00:00 -0800
488: Old ping(8) bug
Finding a 24 year old bug in ping(8), The Role of Operating Systems in IOT, Authentication gateway with SSH on OpenBSD, FreeBSD 12.4 is out, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Fuzzing ping(8) … and finding a 24 year old bug (https://tlakh.xyz/fuzzing-ping.html) The Role of Operating Systems in IOT (https://klarasystems.com/articles/the-role-of-operating-systems-in-iot/) News Roundup Authentication gateway with SSH on OpenBSD (https://dataswamp.org/~solene/2022-12-01-openbsd-authpf.html) FreeBSD 12.4 is out (https://lists.freebsd.org/archives/freebsd-announce/2022-December/000059.html) Beastie Bits Vagrant FreeBSD Boxbuilder (https://github.com/punktDe/vagrant-freebsd-boxbuilder) LibreSSL 3.7.0 Released (https://undeadly.org/cgi?action=article;sid=20221212183516) OPNsense 22.7.9 released (https://opnsense.org/opnsense-22-7-9-released) BIOS Memory Map for vmd(8) Rewrite in Progress (https://undeadly.org/cgi?action=article;sid=20221211164822) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 05 Jan 2023 00:00:00 -0800
487: EuroBSDcon Interviews Pt. 2
This year end episode of BSDNow features a trip report to EuroBSDcon by Mr. BSD.tv, as well as an interview with FreeBSD committer John Baldwin. Happy New Year, 2023! NOTES*** This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) EuroBSDCon 2022 Trip Report (https://freebsdfoundation.org/blog/eurobsdcon-2022-trip-report-patrick-mcevoy/) Interview 3 - John Baldwin - email@email (mailto:email@email) / @twitter (https://twitter.com/user) Interview topic Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 29 Dec 2022 00:00:00 -0800
486: EuroBSDcon interviews
This special episode features two interviews we did at EuroBSDcon in Vienna this year. We talk with FreeBSD developers about how they got started, their current projects and more. Also, consider donating to your favorite BSD Foundation to keep the projects going. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Help the OpenBSD Foundation Reach Its 2022 Funding Goal (http://undeadly.org/cgi?action=article;sid=20221202062601) • [FreeBSD Foundation Donation Link](https://freebsdfoundation.org/donate/) • [NetBSD Foundation Donation Link](http://www.netbsd.org/donations/#how-to-donate) Interview 1 - Brooks Davis - email@email (mailto:email@email) / @twitter (https://twitter.com/user) Interview topic Interview 2 - Olivier Cochard-Labbe - email@email (mailto:email@email) / @twitter (https://twitter.com/user) Interview topic Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. *** Feedback/Questions Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 22 Dec 2022 00:00:00 -0800
485: FreeBSD Home Assistant
Tails of the M1 GPU, Getting Home Assistant running in a FreeBSD 13.1 jail, interview with AWK creator Dr. Brian Kernighan, Next steps toward mimmutable, Unix's (technical) history is mostly old now, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Tails of the M1 GPU (https://asahilinux.org/2022/11/tales-of-the-m1-gpu/) Getting Home Assistant running in a FreeBSD 13.1 jail (https://dan.langille.org/2022/08/27/getting-home-assistant-running-in-a-freebsd-13-1-jail/) News Roundup A brief interview with AWK creator Dr. Brian Kernighan (https://pldb.com/posts/brianKernighan.html) Next steps toward mimmutable, from deraadt@ (https://undeadly.org/cgi?action=article;sid=20221120115616) Unix's (technical) history is mostly old now (https://utcc.utoronto.ca/~cks/space/blog/unix/UnixHistoryMostlyOldNow) MWL Update Fediverse Servers, plus mac_portacl on FreeBSD (https://mwl.io/archives/22392) Fifty Books. Thirty Years. What Next? (https://mwl.io/archives/22399) Mailing List Freebies (https://mwl.io/archives/22423) Beastie Bits More #FreeBSD Power Saving Notes (http://blog.ignoranthack.me/?p=686) Hacker Stations (https://hackerstations.com/) The Cult of DD (https://eklitzke.org/the-cult-of-dd) RavynOS (https://airyx.org/) ravynOS (previously called airyxOS) is an open-source operating system based on FreeBSD, CMU Mach, and Apple open-source code that aims to be compatible with macOS applications and has no hardware restrictions. Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 15 Dec 2022 00:00:00 -0800
484: Birth of stderr
Virtualization showdown, The Birth of Standard Error, why Steam started picking a random font, Maintaining Sufficient Free Space with ZFS, updated Apple M1/M2 bootloader, code, FreeBSD on my workstation, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Virtualization showdown – FreeBSD’s bhyve vs. Linux’s KVM (https://klarasystems.com/articles/virtualization-showdown-freebsd-bhyve-linux-kvm/) The Birth of Standard Error (https://www.spinellis.gr/blog/20131211/) News Roundup Investigating why Steam started picking a random font (http://blog.pkh.me/p/35-investigating-why-steam-started-picking-a-random-font.html) Curious Case of Maintaining Sufficient Free Space with ZFS (https://taras.glek.net/post/curious-case-of-maintaining-sufficient-free-space-with-zfs/) Call for testing on updated Apple M1/M2 bootloader code (https://undeadly.org/cgi?action=article;sid=20221120113149) FreeBSD on my workstation (https://camandro.org/blog/2022-09-30-freebsd-on-my-workstation.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Brad - Initial Setup (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/484/feedback/Brad%20-%20Initial%20Setup.md) Joseph - openbsd and postgresql (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/484/feedback/joseph%20-%20openbsd%20and%20postgresql.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 08 Dec 2022 00:00:00 -0800
483: ZFS Time Machine
Research Unix Version 6 in the Open SIMH PDP-11 Emulator, The Hot Tub Time Machine is Your ZFS Turn-Back-Time Method, NFS on NetBSD: server and client side, HardenedBSD October 2022 Status Report, Nushell : Introduction, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Installing and Using Research Unix Version 6 in the Open SIMH PDP-11 Emulator (http://decuser.blogspot.com/2022/10/installing-and-using-research-unix.html) httm – The Hot Tub Time Machine is Your ZFS Turn-Back-Time Method (https://klarasystems.com/articles/httm-is-a-zfs-based-time-machine/) News Roundup NFS on NetBSD: server and client side (https://www.unitedbsd.com/d/959-nfs-on-netbsd-server-and-client-side) HardenedBSD October 2022 Status Report (https://hardenedbsd.org/article/shawn-webb/2022-10-31/hardenedbsd-october-2022-status-report) Nushell : Introduction (https://dataswamp.org/~solene/2022-10-31-nushell.html) Beastie Bits Unix Pipe Game (https://hackaday.com/2022/10/18/if-only-the-kids-knew-about-pipes/) Slides - The “other” FreeBSD optimizations used by Netflix to serve video at 800Gb/s from a single server (https://people.freebsd.org/~gallatin/talks/euro2022.pdf) My FreeBSD Friday Lecture: The Writing Scholar’s Guide to FreeBSD (https://www.coreystephan.com/freebsd-friday/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Dan - Response to Hans (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/483/feedback/Dan%20-%20Response%20to%20Hans.md) Johnny - bhyve question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/483/feedback/Johnny%20-%20bhyve%20question.md) Manuel - EuroBSDcon social event (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/483/feedback/Manuel%20-%20EuroBSDcon%20social%20event.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 01 Dec 2022 00:00:00 -0800
482: BSD XFCE Desktop
5 Key Reasons to Consider Open Source Storage, OpenBSD Minimalist Desktop, BSD XFCE, Alpine Linux VM on bhyve - with root on ZFS, FreeBSD Jail Quick Setup with Networking, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines 5 Key Reasons to Consider Open Source Storage Over Commercial Offerings (https://klarasystems.com/articles/open-source-storage-over-commercial-offerings/) OpenBSD Minimalist Desktop (https://nechtan.io/articles/openbsd_minimalist_desktop.html) News Roundup BSD-XFCE (https://github.com/Wamphyre/BSD-XFCE) Creating an Alpine Linux VM on bhyve - with root on ZFS (optionally encrypted) (https://it-notes.dragas.net/2022/11/01/creating-an-alpine-vm-on-bhyve-with-root-on-zfs-optionally-encrypted/) FreeBSD Jail Quick Setup with Networking (2022) (https://www.shaka.today/freebsd-jail-quick-setup-with-networking-2022/) Beastie Bits EuroBSDcon videos are now up (https://www.youtube.com/c/EuroBSDcon/videos) LibreSSL 3.6.1 released (https://undeadly.org/cgi?action=article;sid=20221104064712) Raspberry Pi 4 with FreeBSD 13-RELEASE: A Perfect Miniature Homelab (https://www.coreystephan.com/pi4-freebsd/) AsiaBSDcon 2023 CfP (https://2023.asiabsdcon.org/cfp.html.en) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions John - Allan's meetup (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/482/feedback/John%20-%20Allan's%20meetup.md) Matthew - atime and a question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/482/feedback/Matthew%20-%20atime%20and%20a%20question.md) Valentin - Becoming a FreeBSD Developer (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/482/feedback/Valentin%20-%20Becoming%20a%20FreeBSD%20Developer.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 24 Nov 2022 00:00:00 -0800
481: Fiery Crackers
FreeBSD Q3 2022 status report, Leveraging MinIO and OpenZFS to avoid vendor lock in, FreeBSD on Firecracker platform, How Much Faster Is Making A Tar Archive Without Gzip, Postgres from packages on OpenBSD, Upgrading an NVMe zpool from 222G to 1TB drives, Don't use Reddit for Linux or BSD related questions, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD Quarterly Status Report Third Quarter 2022 (https://www.freebsd.org/status/report-2022-07-2022-09/) Avoid Infrastructure Vendor Lock-in by leveraging MinIO and OpenZFS (https://klarasystems.com/articles/avoid-vendor-lock-in-with-minio-and-openzfs/) Announcing the FreeBSD/Firecracker platform (https://www.daemonology.net/blog/2022-10-18-FreeBSD-Firecracker.html) News Roundup How Much Faster Is Making A Tar Archive Without Gzip? (https://lowendbox.com/blog/how-much-faster-is-making-a-tar-archive-without-gzip/) PostgreSQL from packages on OpenBSD (https://www.dbi-services.com/blog/postgresql-from-packages-on-openbsd/) Upgrading an NVMe zpool from 222G to 1TB drives (https://dan.langille.org/2022/10/18/upgrading-an-nvme-zpool-from-222g-to-1tb-drives/) PSA: Don't use Reddit for Linux or BSD related questions (https://unixsheikh.com/articles/dont-use-reddit-for-linux-or-bsd-related-questions.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Hinnerk - vnet jails (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/481/feedback/Hinnerk%20-%20vnet%20jails.md) Tom’s response example: https://adventurist.me/posts/00304 Hugo - Apple M2 (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/481/feedback/Hugo%20-%20Apple%20M2.md) kevin - emacs backspace (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/481/feedback/kevin%20-%20emacs%20backspace.md) ) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 17 Nov 2022 00:00:00 -0800
480: OpenBSD 7.2
OpenBSD 7.2 and FuguIta have been released, Learn the Whys and Hows with the FreeBSD Sec Team, how to get notified about FreeBSD updates, using unbound for ad blocking on OpenBSD, further memory protections on OpenBSD current, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines OpenBSD 7.2 has been released (https://www.openbsd.org/72.html) FuguIta 7.2 is out as well (https://fuguita.org/index.php?FuguIta%2F7.2=) *** ### Keeping FreeBSD Secure: Learn the Whys and Hows with the FreeBSD Sec Team (https://freebsdfoundation.org/blog/keeping-freebsd-secure-learn-the-whys-and-hows-with-the-freebsd-sec-team/) News Roundup Howto: be notified of FreeBSD upgrades, security updates and package updates at login (https://forums.freebsd.org/threads/howto-be-notified-of-freebsd-upgrades-security-updates-and-package-updates-at-login.86660/) Ads blocking with OpenBSD unbound(8) (https://www.tumfatig.net/2022/ads-blocking-with-openbsd-unbound8/) Further memory protections committed to -current (http://undeadly.org/cgi?action=article;sid=20221008100649) Beastie Bits • [“OpenBSD Mastery: Filesystems” Print/Ebook Bundle Preorder](https://mwl.io/archives/22352) • [Klara is hiring a FreeBSD Kernel Developer](https://klarasystems.com/careers/freebsd-kernel-developer/) • [FreeBSD 12.4-BETA1 Now Available](https://lists.freebsd.org/archives/freebsd-stable/2022-October/000920.html) • [Hunting kernel lock and interrupt latency](https://mail-index.netbsd.org/tech-kern/2022/10/30/msg028499.html) • [EuroBSDcon 2022 videos available](https://undeadly.org/cgi?action=article;sid=20221027232308) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Charles - BSD Now Bingo (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/480/feedback/Charles%20-%20BSD%20Now%20Bingo.md) Jake - FreeBSD Security defaults (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/480/feedback/Jake%20-%20FreeBSD%20Security%20defaults.md) Sam - FreeBSD and SSDs (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/480/feedback/Sam%20-%20FreeBSD%20and%20SSDs.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 10 Nov 2022 00:00:00 -0800
479: OpenBSD Docker Host
EuroBSDcon 2022 as first BSD conference, Red Hat’s OpenShift vs FreeBSD Jails, Running a Docker Host under OpenBSD using vmd(8), history of sending signals to Unix process groups, Toolchains adventures - Q3 2022, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines EuroBSDCon 2022, my first BSD conference (and how they are different) (https://eerielinux.wordpress.com/2022/09/25/eurobsdcon-2022-my-first-bsd-conference-and-how-they-are-different/) Red Hat’s OpenShift vs FreeBSD Jails (https://klarasystems.com/articles/red-hats-openshift-vs-freebsd-jails/) News Roundup The history of sending signals to Unix process groups (https://utcc.utoronto.ca/~cks/space/blog/unix/ProcessGroupsAndSignals) Running a Docker Host under OpenBSD using vmd(8) (https://www.tumfatig.net/2022/running-docker-host-openbsd-vmd/) Toolchains adventures - Q3 2022 (https://www.cambus.net/toolchains-adventures-q3-2022/) Beastie Bits -current has moved to 7.2 (https://undeadly.org/cgi?action=article;sid=20220912055003) Several /sbin daemons are now dynamically-linked (http://undeadly.org/cgi?action=article;sid=20220830052924) Announcing the pkgsrc 2022Q3 branch (https://mail-index.netbsd.org/netbsd-announce/2022/09/29/msg000341.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Hans - datacenters and dust (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/476/feedback/Hans%20-%20datacenters%20and%20dust.md) Tim - Boot issue (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/476/feedback/Tim%20-%20Boot%20issue.md) aaron- dwm tiling (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/476/feedback/aaron-%20dwm%20tiling%20.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 03 Nov 2022 00:00:00 -0700
478: Debunking sudo myths
Open Source in Enterprise Environments, Your Comprehensive Guide to rc(8): FreeBSD Services and Automation, How Rob Pike got hired by Dennis Richie, what FreeBSD machines rubenerd uses, new debugbreak command, 7 sudo myths debunked NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Open Source in Enterprise Environments - Where Are We Now and What Is Our Way Forward? (https://bsdly.blogspot.com/2022/09/open-source-in-enterprise-environments.html) Your Comprehensive Guide to rc(8): FreeBSD Services and Automation (https://klarasystems.com/articles/rc8-freebsd-services-and-automation/) News Roundup How Rob Pike got hired by Dennis Richie (https://minnie.tuhs.org/pipermail/tuhs/2022-September/026506.html) Cartron asks what FreeBSD machines I use (https://rubenerd.com/cartron-asks-what-freebsd-machines-i-use/) My new debugbreak command (https://nullprogram.com/blog/2022/07/31/) 7 sudo myths debunked (https://opensource.com/article/22/8/debunk-sudo-myths) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Andy - sharing and acls (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/478/feedback/Andy%20-%20sharing%20and%20acls.md) Reptilicus Rex - boot environments (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/478/feedback/Reptilicus%20Rex%20-%20boot%20environments.md) i3luefire - byhve issue (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/478/feedback/i3luefire%20-%20byhve%20issue.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 27 Oct 2022 00:00:00 -0700
477: Uninitialized Memory Disclosures
Analyzing BSD Kernels for Uninitialized Memory Disclosures Using Binary Ninja, Sharing Dual-Licensed Drivers between Linux and FreeBSD, favorite Things About The OpenBSD Packet Filter Tools, How to trigger services restart after OpenBSD update, Gems from the Man Page Trenches, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Mindshare: Analyzing Bsd Kernels for Uninitialized Memory Disclosures Using Binary Ninja (https://www.zerodayinitiative.com/blog/2022/9/19/mindshare-analyzing-bsd-kernels-with-binary-ninja) Sharing Dual-Licensed Drivers between Linux and FreeBSD (https://freebsdfoundation.org/blog/sharing-dual-licensed-drivers-between-linux-and-freebsd/) News Roundup A Few of My Favorite Things About The OpenBSD Packet Filter Tools (https://nxdomain.no/~peter/better_off_with_pf.html) How to trigger services restart after OpenBSD update (https://dataswamp.org/~solene/2022-09-25-openbsd-reboot-syspatch.html) Gems from the Man Page Trenches (https://www.saminiir.com/gems-from-man-page-trenches/) Beastie Bits The MIPS ThinkPad (https://oldvcr.blogspot.com/2022/09/the-mips-thinkpad-kind-of.html) Nix Gems (https://gitlab.com/DeaDSouL/NixGems) Running PalmOS without PalmOS (https://pmig96.wordpress.com/2022/09/18/running-palmos-without-palmos/) "OpenBSD Mastery: Filesystems" draft done! (https://mwl.io/archives/22303) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Brad - zfs and databases (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/477/feedback/Brad%20-%20zfs%20and%20databases.md) Kevin - EMACS (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/477/feedback/Kevin%20-%20EMACS.md) Michal - virtual OSS (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/477/feedback/Michal%20-%20virtual%20OSS.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 20 Oct 2022 00:00:00 -0700
476: Warren Toomey interview
In this special episode, we interview Warren Toomey from the Unix Historical Society. We chat about his involvement in preserving old Unix systems and why that is important. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Interview - Warren Toomey - wkt@tuhs.org (mailto:wkt@tuhs.org) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** Special Guest: Warren Toomey.
Thu, 13 Oct 2022 00:00:00 -0700
475: Prompt Injection Attacks
Prompt injection attacks against GPT-3, the History of Package Management on FreeBSD, A fresh look at FreeBSD, File Management Tools for Your Favorite Shell, Quick Guide about Video Playback on FreeBSD, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Prompt injection attacks against GPT-3 (https://simonwillison.net/2022/Sep/12/prompt-injection/) A Quick Look at the History of Package Management on FreeBSD (https://klarasystems.com/articles/a-quick-look-at-the-history-of-package-management-on-freebsd/) News Roundup A fresh look at FreeBSD (https://liam-on-linux.dreamwidth.org/86277.html) File Management Tools for Your Favorite Shell (https://thevaluable.dev/file-management-tools-linux-shell/) Video Playback on FreeBSD – Quick Guide (https://freebsdfoundation.org/resource/video-playback-on-freebsd-quick-guide/) Beastie Bits ps(1) gains support for tree-like display of processes (http://undeadly.org/cgi?action=article;sid=20220902085038) ... interesting old-timey UNIXes ... (https://minnie.tuhs.org/pipermail/tuhs/2022-September/026393.html) A retro style online SSH client to play Nethack (https://nethack.glitch.me/?retro=true) The Good, the Bad, and the Ugly: The Unix! Legacy (http://herpolhode.com/rob/ugly.pdf) Game of Trees 0.75 released (http://undeadly.org/cgi?action=article;sid=20220910120430) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Ken - HPR (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/475/feedback/Ken%20-%20HPR.md) Kevin - FreeBSD and EMACS (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/475/feedback/Kevin%20-%20FreeBSD%20and%20EMACS.md) Nathan - Handbook contribution Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/475/feedback/Nathan%20-%20Handbook%20contribution%20Question.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 06 Oct 2022 00:00:00 -0700
474: EuroBSDcon 2022
Deploying FreeBSD on Oracle Cloud, A Tale of 300,000 Imaginary Friends, EuroBSDcon 2022 recap, OpenBSD Mastery: Filesystems” Status Report, OpenBGPD 7.6 Released, immutable userland mappings, Portable OpenSSH commits now SSH-signed, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Deploying FreeBSD on Oracle Cloud (https://klarasystems.com/articles/deploying-freebsd-on-oracle-cloud/) The Things Spammers Believe - A Tale of 300,000 Imaginary Friends (https://bsdly.blogspot.com/2022/09/the-things-spammers-believe-tale-of.html) EuroBSDcon 2022 (https://peter.czanik.hu/posts/eurobsdcon2022/) News Roundup “OpenBSD Mastery: Filesystems” Status Report (https://mwl.io/archives/22031) OpenBGPD 7.6 Released (https://undeadly.org/cgi?action=article;sid=20220916051806) OpenBSD may soon gain further memory protections: immutable userland mappings (http://undeadly.org/cgi?action=article;sid=20220902100648) Portable OpenSSH commits now SSH-signed (https://undeadly.org/cgi?action=article;sid=20220902045137) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 29 Sep 2022 00:15:00 -0700
473: Rusty Kernel Modules
Writing FreeBSD kernel modules in Rust, Details behind the FreeBSD aio LPE, Linux subsystem for FreeBSD, FreeBSD Journal: Science, Systems, and FreeBSD, NetBSD improves Amiga support, OpenBSD on Scaleway Elastic Metal, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Writing FreeBSD Kernel modules in Rust (https://research.nccgroup.com/2022/08/31/writing-freebsd-kernel-modules-in-rust/) Details behind the FreeBSD aio LPE (https://accessvector.net/2022/freebsd-aio-lpe) News Roundup Linux Subsystem for FreeBSD (https://medium.com/nttlabs/linux-subsystem-for-freebsd-500b9a88fda4) FreeBSD Journal: Science, Systems, and FreeBSD (https://freebsdfoundation.org/wp-content/uploads/2022/08/03ae2705ab4362602a6bb90c5b9628c595d8b4fa.2.pdf) NetBSD improves its support for the Commodore Amiga (https://thenewstrace.com/netbsd-an-operating-system-that-is-serious-about-being-cross-platform-now-improves-its-support-for-the-commodore-amiga-1985/243892/) Installing OpenBSD on Scaleway Elastic Metal (https://www.senzilla.io/blog/2022/08/10/installing-openbsd-scaleway-elastic-metal/) Beastie Bits /usr/games removed from the default $PATH (http://undeadly.org/cgi?action=article;sid=20220810120423) How to install and configure mDNSResponder (https://forums.FreeBSD.org/threads/how-to-install-and-configure-mdnsresponder.70713/) How to use consistent exit codes in shell scripts (https://sleeplessbeastie.eu/2022/08/12/how-to-use-consistent-exit-codes-in-shell-scripts) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions [TheHolm - zfs question)[https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/469/feedback/TheHolm%20-%20zfs%20question.md] *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 22 Sep 2022 00:00:00 -0700
472: Consistent Exit Code
FreeBSD on the Framework Laptop, Win32 is the only stable ABI on Linux, why OpenBSD’s documentation is so good, configure dma for mail delivery in jails on internet hosts, introducing muxfs, RAID1C boot support, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD on the Framework laptop (https://xyinn.org/md/freebsd/framework_laptop) Win32 is the only stable ABI on Linux (https://blog.hiler.eu/win32-the-only-stable-abi/) News Roundup Why is the OpenBSD documentation so good? (https://dataswamp.org/~solene/2022-08-18-why-openbsd-documentation-is-good.html) How I configure dma for mail delivery in jails on my internet hosts (https://dan.langille.org/2022/08/15/how-i-configure-dma-for-mail-delivery-in-jails-on-my-internet-hosts/) Introducing muxfs (https://sdadams.org/blog/introducing-muxfs/) RAID 1C boot support added (http://undeadly.org/cgi?action=article;sid=20220813110021) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions [Oliver - shell tip)[https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/469/feedback/Oliver%20-%20shell%20tip.md] Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 15 Sep 2022 00:00:00 -0700
471: De-Penguinization
Ten Things To Do After Installing FreeBSD, BSD for Linux users, r2k22 Hackathon Report on rpki-client, Configuring OpenIKED, De-Penguin Me, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Ten Things To Do After Installing FreeBSD (https://bastillebsd.org/blog/2022/07/14/ten-things-to-do-after-installing-freebsd/) News Roundup hpr3655 :: BSD for Linux users (http://hackerpublicradio.org/eps.php?id=3655) r2k22 Hackathon Report: Job Snijders (job@) on rpki-client and more (http://undeadly.org/cgi?action=article;sid=20220701171631) Configuring OpenIKED (https://wiki.ircnow.org/index.php?n=Iked.Configure) De-Penguin Me (https://depenguin.me/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 08 Sep 2022 00:00:00 -0700
470: 0mp interview
In this special episode, we are interviewing Mateusz Piotrowski about his various roles in the FreeBSD project, his ports work, and a few other interesting things he’s involved with. Enjoy this interview episode, we’ll be back with a regular episode next week. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Interview - Mateusz Piotrowski - 0mp@freebsd.org (mailto:0mp@freebsd.org) / @0mpts (https://twitter.com/0mpts) Interview + BR: Welcome Mateusz. Can you tell our audience a bit about yourself and how you got started with Unix/BSD? + TJ: What can we blame you for (prior/current work, planned projects)? + BR: You served as the first doceng secretary and joined the FreeBSD core team in this term. What interested you in these roles and what do you want to accomplish in this term? + TJ: You are also busy with maintaining some FreeBSD ports. What ports are those? + BR: Can you tell us a bit about your thesis work? + TJ: What does open source work mean for you? + BR: Do you have a cool Unix/BSD tip for us? + TJ: Is there anything else that you'd like to mention before we let you go? Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** Special Guest: Mateusz Piotrowski.
Thu, 01 Sep 2022 00:00:00 -0700
469: Ctrl-C Reset
FreeBSD Q2 2022 Status Report, FreeBSD in Science, fastest yes(1) in the west, Why Programmers Can’t "Reset" Programs With Ctrl-C, Run Slack in FreeBSD’s Linuxulator, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD Q2 2022 Status Report (https://www.freebsd.org/status/report-2022-04-2022-06/) FreeBSD in Science (https://freebsdfoundation.org/blog/guest-post-freebsd-in-science/) News Roundup Fastest yes(1) in the west (https://codegolf.stackexchange.com/questions/199528/fastest-yes-in-the-west/199622#199622) Ctrl-C: Why Programmers Can’t "Reset" Programs With Ctrl-C, but Used to Be Able To, and Why They Should Be Able to Again (https://kevinlawler.com/ctrl-c) Run Slack in FreeBSD’s Linuxulator (https://meka.rs/blog/2022/07/01/freebsd-linuxulator/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 25 Aug 2022 00:00:00 -0700
468: Apples and CHERI
Advocating for FreeBSD in 2022 and Beyond, NetBSD 9.3 released, OPNsense 22.7 available, CHERI-based computer runs KDE for the first time, Run FreeBSD 13.1-RELEASE for ARM64 in QEMU on Apple Silicon Mac, and more Notes This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Advocating for FreeBSD in 2022 and Beyond (https://freebsdfoundation.org/blog/advocating-for-freebsd-in-2022-and-beyond/) NetBSD 9.3 released (http://blog.netbsd.org/tnf/entry/netbsd_9_3_released) News Roundup OPNsense 22.7 released (https://forum.opnsense.org/index.php?topic=29507.0) CHERI-based computer runs KDE for the first time (https://www.theregister.com/2022/07/26/cheri_computer_runs_kde/) Guide: Run FreeBSD 13.1-RELEASE for ARM64 in QEMU on Apple Silicon Mac (https://gist.github.com/ctsrc/a1f57933a2cde9abc0f07be12889f97f) Beastie Bits • [In -current, dhclient(8) now just logs warnings and executes ifconfig(8)](http://undeadly.org/cgi?action=article;sid=20220703114819) • [Freshly installed #NetBSD 4.0.1 booting on a 80386 DX40 with 8MB of RAM in 2022](https://twitter.com/lefinnois/status/1553246084675375104) • [nerdctl](https://twitter.com/woodsb02/status/1554481441060560898?s=28&t=8K7_A1RiWnCDU_Mme4_Yqw) • [Even more Randomness](https://undeadly.org/cgi?action=article;sid=20220731110742) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 18 Aug 2022 00:00:00 -0700
467: Minecraft on NetBSD
Installing BSDs on Cubieboard1, Self-hosting a static site with OpenBSD, httpd, and relayd, NetBSD can also run a Minecraft server, A Little Story About the yes Unix Command, Shell History: Unix, OpenBGPD 7.5 released, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Installing BSDs on Cubieboard1 (https://mekboy.ru/post/bsd-on-cubieboard1.en/) Self-hosting a static site with OpenBSD, httpd, and relayd (https://citizen428.net/blog/self-hosting-static-site-openbsd-httpd-relayd/) News Roundup NetBSD can also run a Minecraft server (https://rubenerd.com/netbsd-can-also-run-a-minecraft-server/) A Little Story About the yes Unix Command (https://endler.dev/2017/yes/) Shell History: Unix (https://portal.mozz.us/gemini/auragem.space/~krixano/ShellHistory-Unix.pdf) OpenBGPD 7.5 released (https://undeadly.org/cgi?action=article;sid=20220716101930) Beastie Bits Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Ludensen - Feedback (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/467/feedback/Ludensen%20-%20Feedback.md) Vidar - OpenRGB (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/467/feedback/Vidar%20-%20OpenRGB.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 11 Aug 2022 00:00:00 -0700
466: cat(1)’s efficiency
Contributing to Open Source Beyond Software Development, bringing TLS 1.3 to the Internet of Old Things, How efficient can cat(1) be, boost the speed of Unix shell programs, Running FreeBSD VNET Jails on AWS EC2 with Bastille, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Contributing to Open Source Beyond Software Development (https://klarasystems.com/articles/contributing-to-open-source-beyond-software-development/) Crypto Ancienne 2.0 now brings TLS 1.3 to the Internet of Old Things (except BeOS) (https://oldvcr.blogspot.com/2022/07/crypto-ancienne-20-now-brings-tls-13-to.html) News Roundup How efficient can cat(1) be? (https://ariadne.space/2022/07/17/how-efficient-can-cat1-be/) Technique significantly boosts the speeds of programs that run in the Unix shell (https://techxplore.com/news/2022-06-technique-significantly-boosts-unix-shell.html) • [binpa.sh](http://binpa.sh/) Running FreeBSD VNET Jails on AWS EC2 with Bastille (https://pertho.net/posts/bastille-vnet-jails-ec2/) Beastie Bits Game of Trees 0.74 released (http://undeadly.org/cgi?action=article;sid=20220720220958) OpenBSD -current has moved to 7.2-beta (https://undeadly.org/cgi?action=article;sid=20220721122727) A Unix Command Line Crash Course (https://itnext.io/unix-command-line-crash-course-453e409d62f5) BSD.DOG vimrc (https://bsd.dog/project/bsd-dog-vimrc/) FreeBSD Speedruns (https://wiki.freebsd.org/Speedruns) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 04 Aug 2022 00:00:00 -0700
465: Deep Space Debugging
Debugging Lisp in Deep Space, 0 Dependency Websites with OpenBSD & AsciiDoc, Deleting old snapshots on FreeBSD, Full multiprocess support in lldb-server, Basic fix between pf tables and macros, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines NASA Programmer Remembers Debugging Lisp in Deep Space (https://thenewstack.io/nasa-programmer-remembers-debugging-lisp-in-deep-space/) 0 Dependency Websites with OpenBSD & AsciiDoc (https://blog.passwordclass.xyz/blogs/2022/06/0-dependency-websites-with-openbsd-asciidoc.html) News Roundup FreeBSD - Deleting old snapshots (https://www.jan0sch.de/post/deleting-old-zfs-snapshots/) Full multiprocess support in lldb-server (https://www.moritz.systems/blog/full-multiprocess-support-in-lldb-server/) Basic fix between pf tables and macros on FreeBSD (https://rubenerd.com/basic-fix-between-pf-tables-and-macros-on-freebsd/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Ben - Jail Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/464/feedback/Ben%20-%20Jail%20Question.md) Malcolm - encryption (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/464/feedback/Malcolm%20-%20encryption.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 28 Jul 2022 00:00:00 -0700
464: Compiling with kefir
From 0 to bhyve on FreeBSD, Analyze OpenBSD’s Kernel with Domain-Specific Knowledge, OpenBSD Webzine: ISSUE #10, HardenedBSD June 2022 Status Report, two new C compilers: chibicc and kefir in OpenBSD, SSD TRIM in NetBSD HEAD, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines From 0 to Bhyve on FreeBSD 13.1 (https://klarasystems.com/articles/from-0-to-bhyve-on-freebsd-13-1/) Analyze OpenBSD’s Kernel with Domain-Specific Knowledge (https://medium.com/@chrissicool/analyze-openbsds-kernel-with-domain-specific-knowledge-ca665d92eebb) News Roundup OpenBSD Webzine: ISSUE #10 (https://webzine.puffy.cafe/issue-10.html) HardenedBSD June 2022 Status Report (https://hardenedbsd.org/article/shawn-webb/2022-06-28/hardenedbsd-june-2022-status-report) OpenBSD has two new C compilers: chibicc and kefir (https://briancallahan.net/blog/20220629.html) SSD TRIM in NetBSD HEAD (-current) (https://www.unitedbsd.com/d/859-ssd-trim-in-netbsd-head-current) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 21 Jul 2022 00:00:00 -0700
463: The 1.0 Legend
Differences between base and ports LLVM in OpenBSD, Netgraph for FreeBSD’s bhyve Networking, Audio on FreeBSD – Quick Guide, FreeBSD’s Legend starts at 1.0, Hacker News running by FreeBSD, TrueNAS 13, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Differences between base and ports LLVM in OpenBSD (https://www.cambus.net/differences-between-base-and-ports-llvm-in-openbsd/) Using Netgraph for FreeBSD’s bhyve Networking (https://klarasystems.com/articles/using-netgraph-for-freebsds-bhyve-networking/?utm_source=bsdweekly) News Roundup Audio on FreeBSD – Quick Guide (https://freebsdfoundation.org/freebsd-project/resources/audio-on-freebsd/) [Legends start at 1.0! – FreeBSD in 1993] Part 1 (https://eerielinux.wordpress.com/2022/06/18/legends-start-at-1-0-freebsd-in-1993-pt-1/) Part 2 (https://eerielinux.wordpress.com/2022/06/19/legends-start-at-1-0-freebsd-in-1993-pt-2/) *** ### Hacker News running by FreeBSD. Take that, Linux! (https://news.ycombinator.com/item?id=16076041) *** ### TrueNAS 13 (https://www.theregister.com/2022/05/11/truenas_13_released/) *** Beastie Bits Notable OpenBSD news you may have missed, 2022-06-28 edition (http://undeadly.org/cgi?action=article;sid=20220628135253) rEFInd design for all the BSDs (https://github.com/indgy/refind-bsd-black) OpenBGPD 7.4 released (https://undeadly.org/cgi?action=article;sid=20220619185920) Hotfix GhostBSD 22.06.18 ISO is now available (http://ghostbsd.org/22.06.18_iso_is_now_available) *** ###Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Brad - Jails Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/463/feedback/Brad%20-%20Jails%20Question.md) Freezr - A few questions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/463/feedback/Freezr%20-%20A%20few%20questions.md) A different Brad - Drive question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/463/feedback/A%20different%20Brad%20-%20Drive%20question.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 14 Jul 2022 00:00:00 -0700
462: OpenBSD Sales Pitch
The Design and Implementation of the NetBSD rc.d system, selling OpenBSD as a salesperson, Speeding up autoconf with caching, Allowing non-root execution of a jailed application, Configure login(1) and sshd(8) for YubiKey on OpenBSD, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines The Design and Implementation of the NetBSD rc.d system (http://www.mewburn.net/luke/papers/rc.d.pdf) How I would sell OpenBSD as a salesperson (https://dataswamp.org/~solene/2022-06-22-openbsd-selling-arguments.html) News Roundup Speeding up autoconf with caching (https://jmmv.dev/2022/06/autoconf-caching.html) Allowing non-root execution of a jailed application (https://forums.freebsd.org/threads/allowing-non-root-execution-of-a-jailed-application.85532/) Configure login(1) and sshd(8) for YubiKey on OpenBSD (https://romanzolotarev.com/openbsd/yubikey.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Glen - Thanks Todd (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/462/feedback/Glen%20-%20Thanks%20Todd.md) Karl - Memory Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/462/feedback/Karl%20-%20Memory%20Question.md) alejandro - Tom's laptop (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/462/feedback/alejandro%20-%20Tom's%20laptop.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 07 Jul 2022 00:00:00 -0700
461: Persistent Memory Allocation
Q1 FreeBSD Quarterly Status Report 2022, Nginx on OpenBSD 7.1, Persistent Memory Allocation, Colorize your BSD shell, cgit With Gitolite and Nginx on FreeBSD 13, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD Quarterly Status Report First Quarter 2022 (https://www.freebsd.org/status/report-2022-01-2022-03/) Installing Nginx on OpenBSD 7.1 (https://unixcop.com/installing-nginx-on-openbsd-7-1/) News Roundup Live Webinar: Open-source Virtualization: Getting started with bhyve (https://klarasystems.com/webinars/live-sessions-singup/webinar-open-source-virtualization-getting-started-with-bhyve/) Hosted by Jim Salter and Allan Jude Live July 12th at 13:00 ET Available on-demand a few days later Persistent Memory Allocation (https://queue.acm.org/detail.cfm?id=3534855) Colorize your BSD shell (https://forums.FreeBSD.org/threads/colorize-your-bsd-shell.85458/) How to Install cgit With Gitolite and Nginx on FreeBSD 13 (https://herrbischoff.com/2021/10/how-to-install-cgit-with-gitolite-and-nginx-on-freebsd-13) EuroBSDCon 2022 (Austria) Program announced (https://2022.eurobsdcon.org/program/) Come to Austria and learn about the latest happenings in the BSDs 2 days of tutorials, and 2 days of 3 concurrent tracks of talks Registration is open now. See you there! *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Brad - Drive question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/461/feedback/Brad%20-%20Drive%20question.md) Carl - Wiring question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/461/feedback/Carl%20-%20Wiring%20question.md) Jon - Jails question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/461/feedback/Jon%20-%20Jails%20question.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 30 Jun 2022 00:45:00 -0700
460: OpenBSD airport folklore
Containerd gains support for launching Linux containers on FreeBSD, OpenBSD 7.1 on PINE64 RockPro64, true minimalistic window manager does not exist, OpenBSD folklore, HardenedBSD May 2022 Status Report, DragonFlyBSD 6.2.2 out, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Containerd gains support for launching Linux containers on FreeBSD (https://github.com/containerd/containerd/pull/7000) Uses Linux compat and the Linux Jails concept to deploy a full Linux container userland on FreeBSD OpenBSD 7.1 on PINE64 RockPro64 (https://bsandro.tech/posts/openbsd-7.1-on-pine64-rockpro64/) News Roundup Live Webinar: Open-source Virtualization: Getting started with bhyve (https://klarasystems.com/webinars/live-sessions-singup/webinar-open-source-virtualization-getting-started-with-bhyve/) Hosted by Jim Salter and Allan Jude Live July 12th at 13:00 ET Available on-demand a few days later The True Minimalistic Window Manager Does Not Exist (https://serhanekici.com/ttmwm.html) OpenBSD folklore and share/misc/airport (https://www.cambus.net/openbsd-folklore-and-share-misc-airport/) HardenedBSD May 2022 Status Report (https://hardenedbsd.org/article/shawn-webb/2022-06-01/hardenedbsd-may-2022-status-report) DragonFlyBSD 6.2.2 out (https://www.dragonflydigest.com/2022/06/10/27047.html) Changelog (https://lists.dragonflybsd.org/pipermail/commits/2022-June/820953.html) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Norbert - question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/460/feedback/Norbert%20-%20question.md) Paulo - network question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/460/feedback/Paulo%20-%20network%20question.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 23 Jun 2022 00:00:00 -0700
459: NetBSD Kernel benchmark
Evaluating FreeBSD CURRENT for Production Use, Time Machine-like Backups on OpenBSD, FreeBSD on the Graviton 3, Compiling the NetBSD kernel as a benchmark, Network Management with the OpenBSD Packet Filter Toolset from BSDCan 2022, Hardware Detection & Diagnostics for New FreeBSD Users, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Evaluating FreeBSD CURRENT for Production Use (https://klarasystems.com/articles/evaluating-freebsd-current-for-production-use/) Time Machine like Backups on OpenBSD (https://xosc.org/timemachine.html) News Roundup FreeBSD on the Graviton 3 (https://www.daemonology.net/blog/2022-05-23-FreeBSD-Graviton-3.html) Compiling the NetBSD kernel as a benchmark (https://blog.anotherhomepage.org/post/2022/05/25/Compiling-the-NetBSD-kernel-as-a-benchmark/) Network Management with the OpenBSD Packet Filter Toolset from BSDCan 2022 (http://undeadly.org/cgi?action=article;sid=20220607112236) Hardware Detection & Diagnostics for New FreeBSD Users & PCs (https://forums.FreeBSD.org/threads/hardware-detection-diagnostics-for-new-freebsd-users-pcs.84596/) Beastie Bits • [NetBSD - Announcing Google Summer of Code 2022 projects](https://blog.netbsd.org/tnf/entry/announcing_google_summer_of_code3) • [Welcome FreeBSD Google Summer of Code Participants](https://freebsdfoundation.org/blog/welcome-freebsd-google-summer-of-code-participants/) • [Network from Scratch](https://www.networksfromscratch.com) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 16 Jun 2022 00:00:00 -0700
458: Traceroute interpretation
Fundamentals of the FreeBSD Shell, Spammers in the Public Cloud, locking user accounts properly, overgrowth on NetBSD, moreutils, ctwm & spleen, interpreting a traceroute, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Fundamentals of the FreeBSD Shell (https://klarasystems.com/articles/interacting-with-freebsd-learning-the-fundamentals-of-the-freebsd-shell-2/) Spammers in the Public Cloud, Protected by SPF; Intensified Password Groping Still Ongoing; Spamware Hawked to Spamtraps (https://bsdly.blogspot.com/2022/04/spammers-in-public-cloud-protected-by.html) News Roundup A cautionary tale about locking Linux & FreeBSD user accounts (https://www.cyberciti.biz/networking/a-cautionary-tale-about-locking-linux-freebsd-user-accounts/) Overgrowth runs on NetBSD (https://www.reddit.com/r/openbsd_gaming/comments/ucgavg/i_was_able_to_build_overgrowth_on_netbsd/) moreutils (https://joeyh.name/code/moreutils/) NetBSD, CTWM, and Spleen (https://www.cambus.net/netbsd-ctwm-and-spleen/) How to properly interpret a traceroute or mtr (https://phil.lavin.me.uk/2022/03/how-to-properly-interpret-a-traceroute-or-mtr/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Lets talk a bit about some of the events happening this year, BSDCan in virtual this weekend, emfcamp is this weekend too and in person, MCH is this summer and eurobsdcon is in september. How were the postgres conferences benedict? Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 09 Jun 2022 00:00:00 -0700
457: The NetBSD Wheelbarrow
Journey to ZFS RAIDZ1 on NetBSD, FreeBSD networking basics: WiFi and Bluetooth, smuggling code into the playstation via NetBSD driver hole, KDE FreeBSD CI, remembering buildtool, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines The journey to ZFS raidz1 with different sized disks (On NetBSD) (Wheelbarrow optional) (http://netbsd0.blogspot.com/2022/05/the-journey-to-zfs-raidz1-with.html) FreeBSD Networking Basics: WiFi and Bluetooth (https://freebsdfoundation.org/freebsd-project/resources/networking-basics-wifi-and-bluetooth/) News Roundup Playstation: Hole in NetBSD driver could allow code smuggling (https://www.kiratas.com/playstation-hole-in-netbsd-driver-could-allow-code-smuggling-2/) Archive link if the page is down (no images) (https://web.archive.org/web/20220519162432/https://www.kiratas.com/playstation-hole-in-netbsd-driver-could-allow-code-smuggling-2/) Original Announcment (https://hackerone.com/reports/1350653) German Article (https://www.heise.de/news/Playstation-Luecke-in-NetBSD-Treiber-koennte-Codeschmuggel-ermoeglichen-7091153.html) KDE-FreeBSD CI (https://euroquis.nl//kde/2022/04/26/freebsd-ci.html) Remembering Buildtool (https://jmmv.dev/2022/05/remembering-buildtool.html) Beastie Bits By the Way... Kubernetes for FreeBSD (https://medium.com/@norlin.t/by-the-way-kubernetes-for-freebsd-d0ba4dab8d8e) FreeBSD Games Directory (https://github.com/tigersharke/FreeBSD-Games-Directory) Candlelit Console patch set to the framebuffer console (http://undeadly.org/cgi?action=article;sid=20220516093712) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Dan - A couple things (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/457/feedback/Dan%20-%20A%20couple%20things.md) Paul - BSD Business Justifications (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/457/feedback/Paul%20-%20BSD%20Business%20Justifications.md) Todd - Feedback to prior feedback (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/457/feedback/Todd%20-%20Feedback%20to%20prior%20feedback.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 02 Jun 2022 00:00:00 -0700
456: FreeBSD 13.1
FreeBSD 13.1 is released, Unix command line conventions over time, Branching for NetBSD 10, Microbhyve, Own your Calendar and Contacts with OpenBSD, the PSARC case for ZFS, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD 13.1 Release is available (https://www.freebsd.org/releases/13.1R/announce/) Unix command line conventions over time (https://blog.liw.fi/posts/2022/05/07/unix-cli/) News Roundup Branching for NetBSD 10 (https://mail-index.netbsd.org/current-users/2022/05/02/msg042278.html) Microbyhve (https://github.com/cbsd/microbhyve) Own Your Calendar & Contacts With OpenBSD, Baïkal, and FOSS Android (https://baak6.com/baikal-openbsd-fossdroid/) Twenty years ago today, Jeff filed the PSARC case for the ZFS filesystem (https://twitter.com/mmusante/status/1518947283626246145?t=tzR6KeMx2mhjJfeoOqrHIw&s=03) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Scott - FreeBSD and supercomputing (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/456/feedback/Scott%20-%20FreeBSD%20and%20supercomputing.md) Nick - Thanks and some shout outs (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/456/feedback/Nick%20-%20Thanks%20and%20some%20shout%20outs.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 26 May 2022 00:00:00 -0700
455: Ken Thompson Singularity
OpenBSD is the Perfect OS post Nuclear Apocalypse, Multiprocess support for LLDB, porting the new Hare compiler to OpenBSD, Writing my first OpenBSD game using Godot, FreeBSD 13 on Thinkpad T460s, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines OpenBSD is the Perfect OS post Nuclear Apocalypse (https://confuzeus.com/shorts/openbsd-nuclear-apocalypse/) Multiprocess support for LLDB (https://www.moritz.systems/blog/multiprocess-support-for-lldb/) News Roundup I ported the new Hare compiler to OpenBSD (https://briancallahan.net/blog/20220427.html) Writing my first OpenBSD game using Godot (https://dataswamp.org/~solene/2022-04-28-writing-a-game-with-godot.html) FreeBSD 13 on Thinkpad T460s (https://www.tumfatig.net/2022/freebsd-13-on-thinkpad-t460s/) Beastie Bits Open Source Voices interview with Deb Goodkin (https://www.opensourcevoices.org/29) Tachyum Successfully Runs FreeBSD in Prodigy Ecosystem, Expands Open-Source OS Support (https://www.hpcwire.com/off-the-wire/tachyum-successfully-runs-freebsd-in-prodigy-ecosystem-expands-open-source-os-support/) MidnightBSD Minor Update 2.1.7 (https://midnightbsd.org/security/index.html#a20220404) LibreSSL 3.5.2 Released (https://bsdsec.net/articles/libressl-3-5-2-released) OpenBGPD 7.3 is out (https://undeadly.org/cgi?action=article;sid=20220414091532) Playing the game Bottomless on OpenBSD (https://videos.pair2jeux.tube/w/jheVDTPmBTQzkmSpNSvk8J) Windows Central: OpenBSD already has a version for Apple Silicon (https://windows11central.com/en/openbsd-already-has-a-version-for-apple-silicon/) OpenBSD Webzine #9 is out (https://webzine.puffy.cafe/issue-9.html) In the "Everone makes mistakes catagory" : I forgot to enable compression on ZFS (https://dan.langille.org/2022/04/28/i-forgot-to-enable-compression-on-zfs/) "Ken Thompson is a singularity" ~Brian Kernighan (https://www.youtube.com/watch?v=fL2QwyxcJ5s) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Ben - Securing FreeBSD (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/455/feedback/Ben%20-%20Securing%20FreeBSD.md) Dave - BSD certifications (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/455/feedback/Dave%20-%20BSD%20certifications.md) Sam - maintaining a port (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/455/feedback/Sam%20-%20maintaining%20a%20port.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 19 May 2022 00:00:00 -0700
454: Compiling 50% faster
OpenBSD 7.1 is out, Building Your Own FreeBSD-based NAS with ZFS Part 2, Let's try V on OpenBSD, Waiting for Randot, Compiling an OpenBSD kernel 50% faster, A Salute for 10+ years of service, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines OpenBSD 7.1 is out (https://www.openbsd.org/71.html) Building Your Own FreeBSD-based NAS with ZFS Part 2 (https://klarasystems.com/articles/part-2-tuning-your-freebsd-configuration-for-your-nas/) News Roundup Let's try V on OpenBSD (https://briancallahan.net/blog/20220426.html) Waiting for Randot (or: nia and maya were right and I was wrong) (http://mail-index.netbsd.org/tech-security/2021/01/11/msg001100.html) Compiling an openbsd kernel 50% faster (https://flak.tedunangst.com/post/compiling-an-openbsd-kernel-50-faster) A Salute for 10+ years of service (http://aboutbsd.net/?page_id=26661) https://archive.ph/JL5hf (if the site is down) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Glenn - Toms Home Lab (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/454/feedback/Glenn%20-%20Toms%20Home%20Lab.md) Iamchunky_pie - unix tool writing (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/454/feedback/I_am_chunky_pie%20-%20unix%20tool%20writing.md) Mike - Making Routers (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/454/feedback/Mike%20-%20Making%20Routers.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 12 May 2022 00:00:00 -0700
453: TwinCat/BSD Hypervisor
Building Your Own FreeBSD-based NAS, Writing a device driver for Unix V6, EC2: What Colin Percival’s been up to, Beckhoff releases TwinCAT/BSD Hypervisor, Writing a NetBSD kernel module, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Building Your Own FreeBSD-based NAS (https://klarasystems.com/articles/building-your-own-freebsd-based-nas-with-zfs/) Writing a device driver for Unix V6 (https://mveg.es/posts/writing-a-device-driver-for-unix-v6/) News Roundup FreeBSD/EC2: What I've been up to (https://www.daemonology.net/blog/2022-03-29-FreeBSD-EC2-report.html) Beckhoff has released its TwinCAT/BSD Hypervisor (https://www.automationworld.com/control/article/22144694/beckhoff-hypervisor-enables-virtual-machines-for-control-applications) Writing a NetBSD kernel module (https://saurvs.github.io/post/writing-netbsd-kern-mod/) Benedicts Git Finds Projects Run anything (like full blown GTK apps) under Capsicum (https://github.com/unrelentingtech/capsicumizer) Twitter client for UEFI (https://github.com/arata-nvm/mitnal) n³ The unorthodox terminal file manager (https://github.com/jarun/nnn) OpenVi: Portable OpenBSD vi for UNIX systems (https://github.com/johnsonjh/OpenVi) Gists and Articles Step-by-step instructions on installing the latest NVIDIA drivers on FreeBSD 13.0 and above (https://gist.github.com/Mostly-BSD/4d3cacc0ee2f045ed8505005fd664c6e) FreeBSD SSH Hardening (https://gist.github.com/koobs/e01cf8869484a095605404cd0051eb11) GTFOBins is a curated list of Unix binaries that can be used to bypass local security restrictions in misconfigured systems (https://gtfobins.github.io) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Ben - Backing Up (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/453/feedback/Ben%20-%20Backing%20Up.md) Ethan - Thanks (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/453/feedback/Ethan%20-%20Thanks.md) Maxi - question about note taking (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/453/feedback/Maxi%20%20-%20question%20about%20note%20taking.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 05 May 2022 00:00:00 -0700
452: The unknown hackers
The unknown hackers, Papers we love to read, Dual Boot Homelab in The Bedroom by the bed testbed, OpenSSH 9.0 released, OS battle: OpenBSD vs. NixOS, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines The unknown hackers (https://www.salon.com/2000/05/17/386bsd/) Bill Jolitz passed away in March 2022 (https://minnie.tuhs.org/pipermail/tuhs/2022-April/025643.html) *** FreeBSD Documentation: Papers We Love To Read (https://klarasystems.com/articles/freebsd-documentation-papers-we-love-to-read/) News Roundup FreeBSD/Ubuntu Dual Boot Homelab in The Bedroom by the bed testbed (https://adventurist.me/posts/00307) OpenSSH 9.0 has been released (https://www.openssh.com/txt/release-9.0) Operating systems battle: OpenBSD vs NixOS (https://dataswamp.org/~solene/2022-04-18-openbsd-vs-nixos.html) Beastie Bits Celebrating 50 years of the Unix Operating System (https://www.reddit.com/r/BSD/comments/u4t25c/celebrating_50_years_of_the_unix_operating_system/) Kickstarter Campaign Results (https://mwl.io/archives/13627) FreeBSD Virtualization Series (https://productionwithscissors.run/freebsd-virtualization-series/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Jeff - ZFS checksum repair (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/452/feedback/Jeff%20-%20ZFS%20checksum%20repair.md) Nelson - General Thanks (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/452/feedback/Nelson%20-%20General%20Thanks.md) Sam - FOSS Power Support (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/452/feedback/Sam%20-%20FOSS%20Power%20Support.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 28 Apr 2022 00:00:00 -0700
451: Tuning ZFS recordsize
Full system backups with FFS snapshots, ZFS and dump(8), tuning recordsize in OpenZFS, Optimizing FreeBSD Power Consumption on Modern Intel Laptops, remember to check for ZFS filesystems being mounted, Use tcpdump to save wireless bridge, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Full system backups with FFS snapshots, ZFS and dump(8) (https://www.unitedbsd.com/d/705-full-system-backups-with-ffs-snapshots-zfs-and-dump8) Tuning Recordsize in OpenZFS (https://klarasystems.com/articles/tuning-recordsize-in-openzfs/) News Roundup Optimizing FreeBSD Power Consumption on Modern Intel Laptops (https://www.neelc.org/posts/optimize-freebsd-for-intel-tigerlake/) I need to remember to check for ZFS filesystems being mounted (https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSCheckForMounted) Use tcpdump to save wireless bridge (https://adventurist.me/posts/0027) Beastie Bits • [FreeBSD on the Vortex86DX CPU](https://www.cambus.net/freebsd-on-the-vortex86dx-cpu/) • [HAMMER2 vs USB stick pulls](https://www.dragonflydigest.com/2022/03/22/26800.html) • [New US mirror for DragonFly](https://www.dragonflydigest.com/2022/03/09/26742.html) • [HelloSystem 13.1 RC1](https://github.com/helloSystem/ISO/releases/tag/experimental-13.1-RC1) • [Video introduction to OpenBSD 7.0](https://www.youtube.com/watch?v=KeUsE-3nSes) • [Losses in the community](https://minnie.tuhs.org/pipermail/tuhs/2022-April/025643.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Sam - BSD Laptops (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/451/feedback/Sam%20-%20BSD%20Laptops.md) Reese - Electric Groff (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/451/feedback/Reese%20-%20Electric%20Groff.md) Alexandra - New to BSD (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/451/feedback/Alexandra%20-%20New%20to%20BSD.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 21 Apr 2022 00:00:00 -0700
450: Unix Tool Writing
The ideas that made Unix, hints for writing Unix tools, cron best practices, three different sorts of filesystem errors, LibreSSL 3.5.1 released, taskwarrior to manage tasks, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Unix Philosophy: A Quick Look at the Ideas that Made Unix (https://klarasystems.com/articles/unix-philosophy-a-quick-look-at-the-ideas-that-made-unix/) Hints for writing Unix Tools (https://monkey.org/~marius/unix-tools-hints.html) News Roundup Cron best practices (https://blog.sanctum.geek.nz/cron-best-practices/) Filesystems can experience at least three different sorts of errors (https://utcc.utoronto.ca/~cks/space/blog/tech/FilesystemsThreeErrorTypes) LibreSSL 3.5.1 development branch as well as 3.4.3 (stable) and 3.3.6 released (https://undeadly.org/cgi?action=article;sid=20220318065203) Taskwarrior to manage tasks (https://adventurist.me/posts/0165) Beastie Bits Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Andrew - virtualization (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/450/feedback/Andrew%20-%20virtualization.md) Brad - jails applications and interoperability (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/450/feedback/brad%20-%20jails%20applications%20and%20interoperability.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 14 Apr 2022 00:00:00 -0700
449: Reproducible clean $HOME
FreeBSD Status Report 4th Quarter 2021, Reproducible clean $HOME in OpenBSD using impermanence, Making RockPro64 a NetBSD Server, helloSystem 0.7.0 is out, lazy approach to FreeBSD dual-booting, going to jail, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD Quarterly Status Report 4th Quarter 2021 (https://www.freebsd.org/status/report-2021-10-2021-12/) Reproducible clean $HOME in OpenBSD using impermanence (https://dataswamp.org/~solene/2022-03-15-openbsd-impermanence.html) News Roundup Making RockPro64 a NetBSD Server (https://blog.netbsd.org/tnf/entry/making_rockpro64_a_netbsd_server) helloSystem 0.7.0 is out (https://github.com/helloSystem/ISO/releases/tag/r0.7.0) My lazy approach to FreeBSD dual-booting (https://rubenerd.com/my-lazy-approach-to-freebsd-dual-booting/) Going to jail (https://opekkt.tech/docs/vps_migration/going2jail/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions • No Feedback emails this week, so instead we can have “Story Time with Allan” and he can regale us with an entertaining BSD story. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 07 Apr 2022 00:00:00 -0700
448: Controlling Resource Limits
Controlling Resource Limits with rctl in FreeBSD, It’s always DNS, Google Summer of Code in BSD Projects, Rsync Technical Notes - Q4 2021, Userland CPU frequency scheduling for OpenBSD, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Controlling Resource Limits with rctl in FreeBSD (https://klarasystems.com/articles/controlling-resource-limits-with-rctl-in-freebsd/) It's DNS. Of course it's DNS, it's always DNS. (https://utcc.utoronto.ca/~cks/space/blog/sysadmin/DNSVariabilityProblems) News Roundup GSOC • [Work with FreeBSD in Google Summer of Code](https://freebsdfoundation.org/blog/work-with-freebsd-in-google-summer-of-code/) • [The NetBSD Foundation is a mentoring organization at Google Summer of Code 2022](https://blog.netbsd.org/tnf/entry/the_netbsd_foundation_is_a) Rsync Technical Notes - Q4 2021 (https://www.rsync.net/resources/notes/2021-q4-rsync.net_technotes.html) Userland CPU frequency scheduling for OpenBSD (https://tildegit.org/solene/obsdfreqd) Beastie Bits Unofficial HardenedBSD liveCD (https://groups.google.com/a/hardenedbsd.org/g/users/c/QUTUJfm30Dg/m/0VNKUeVhHgAJ) The eurobsdcon 2022 CFP is open (https://2022.eurobsdcon.org/the-call-for-talk-and-presentation-proposals-for-eurobsdcon-2022-is-now-open/) Testing parallel forwarding (http://undeadly.org/cgi?action=article;sid=20220319123157) OpenBSD iwx(4) gains 11ac 80MHz channel support (https://www.undeadly.org/cgi?action=article;sid=20220315070043) OpenBSD/arm64 on Apple M1 systems (https://undeadly.org/cgi?action=article;sid=20220320115932) FreeBSD on the CubieBoard2 (https://www.cambus.net/freebsd-on-the-cubieboard2/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Eric - periodic notifications (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/448/feedback/Eric%20-%20periodic%20notifications.md) Kevin - no question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/448/feedback/Kevin%20-%20no%20question.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 31 Mar 2022 00:00:00 -0700
447: Path to BSD
FreeBSD Foundation Proposals, UNIX: On the Path to BSD, Fujitsu ends its mainframe and Unix services, Install burpsuite on FreeBSD using Linuxulator, new OpenBSD Webzine is out, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Project Proposal Overview (https://freebsdfoundation.org/get-involved/project-proposal-overview/) UNIX: On the Path to BSD (https://klarasystems.com/articles/unix-on-the-path-to-bsd/) News Roundup Fujitsu is ending its mainframe and Unix services (https://www.techradar.com/in/news/fujitsu-is-ending-its-mainframe-and-unix-services) TUTORIAL: Install burpsuite on FreeBSD using Linuxulator (https://forums.FreeBSD.org/threads/tutorial-install-burpsuite-on-freebsd-using-linuxulator.84310/) OpenBSD Webzine (https://webzine.puffy.cafe/issue-7.html) Beastie Bits A Trio if OPNsense releases: 21.7.8 (https://opnsense.org/opnsense-21-7-8-released/) 21.10.3 (https://opnsense.org/opnsense-business-edition-21-10-3-released/) 22.1.1 (https://opnsense.org/opnsense-22-1-1-released-2) FreeBSD 12.2 end-of-life (https://lists.freebsd.org/archives/freebsd-announce/2022-March/000018.html) DragonFly as a KVM guest (https://www.dragonflybsd.org/docs/howtos/HowToKvmGuest/) RIP Lorinda Cherry (https://lwn.net/ml/tuhs/CAKH6PiVi+JoxDG7ACMG5G+qnTkxTMsohGx6Wq3UNVkogO4N0Vg@mail.gmail.com/) Precursor: From Boot to Root (https://www.bunniestudios.com/blog/?p=6336) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions No Feedback emails this week, so instead Tom can regale us with an entertaining BSD story. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 24 Mar 2022 00:00:00 -0700
446: Debugging ioctl problems
Restoring a Tadpole SPARCbook 3, The FreeBSD Boot Process, Debugging an ioctl Problem on OpenBSD, Why my game PC runs FreeBSD and Kubuntu, DNSSEC, Badgers, and Orcs, Oh My, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Restoring a Tadpole SPARCbook 3 Part 1: Introduction (https://www.rs-online.com/designspark/restoring-a-tadpole-sparcbook-3-part-1-introduction) The FreeBSD Boot Process (https://klarasystems.com/articles/the-freebsd-boot-process/) News Roundup Debugging an ioctl Problem on OpenBSD (https://jcs.org/2022/02/16/ioctl) Why my game PC runs FreeBSD and Kubuntu (https://rubenerd.com/why-my-game-pc-also-runs-freebsd/) DNSSEC, Badgers, and Orcs, Oh My! (https://mwl.io/archives/14708) Beastie Bits • [LibreSSL 3.5.0 development branch released](https://undeadly.org/cgi?action=article;sid=20220301063844) • [OpenSSH updated to 8.9](https://undeadly.org/cgi?action=article;sid=20220301063428) • [Recent developments in OpenBSD, 2022-02-21 summary](https://undeadly.org/cgi?action=article;sid=20220221060700) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Jonathan - X-Wing and Tie Fighter (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/446/Jonathan%20-%20X-Wing%20and%20Tie%20Fighter.md) Joshontech - pool options (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/446/joshontech%20-%20pool%20options.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 17 Mar 2022 00:00:00 -0700
445: Journey to BSD
Idiot's guide to OpenBSD on the Pinebook Pro, FreeBSD Periodic Scripts, history of service management in Unix, journey from macOS to FreeBSD, Unix processes “infecting” each other, navidrom music server on FreeBSD, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines The complete idiot's guide to OpenBSD on the Pinebook Pro (https://tomscii.sig7.se/2022/02/Guide-to-OpenBSD-on-the-PinebookPro) FreeBSD Periodic Scripts (https://klarasystems.com/articles/freebsd-periodic-scripts/) News Roundup The history (sort of) of service management in Unix (https://utcc.utoronto.ca/~cks/space/blog/unix/ServiceManagementHistory) My journey from macOS to FreeBSD (https://www.boucek.me/blog/from-mac-to-freebsd/) A nice story about Unix processes "infecting" each other (https://rachelbythebay.com/w/2022/02/09/nice/) Navidrome music server on FreeBSD (https://web.archive.org/web/20220101220446/https://www.danschmid.me/article/install-navidrome-music-server-on-freebsd) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Tyler - Is this enough for VMs (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/444/feedback/Tyler%20-%20Is%20this%20enough%20for%20VMs.md) Kevin - BSD from RAMdisk (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/445/feedback/Kevin%20-%20BSD%20from%20RAMdisk.md) Malcolm - wired headset in FreeBSD (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/445/feedback/Malcolm%20-%20wired%20headset%20in%20FreeBSD.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 10 Mar 2022 00:00:00 -0800
444: Historic Developments
The History of Berkeley DB, modern inetd in FreeBSD, the Unix argv[0] issue, retrocomputing can be more than games, read section 8 of the Unix users manual, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines A Conversation with Margo Seltzer and Mike Olson: The history of Berkeley DB (https://queue.acm.org/detail.cfm?id=3501713) Modern inetd in FreeBSD (https://klarasystems.com/articles/modern-inetd-in-freebsd/) News Roundup The reason Unix has the argv[0] issue (and API) (https://utcc.utoronto.ca/~cks/space/blog/unix/Argv0IsEasy) Retrocomputing can be more than games (https://rubenerd.com/retrocomputing-is-more-than-games/) You should read Section 8 of the Unix User's Manual (https://www.theregister.com/2022/02/09/section_8_unix_user_manual/) Beastie Bits New 'Reckless guide to OpenBSD' published (https://undeadly.org/cgi?action=article;sid=20220214061716) GhostBSD Online Meetup (http://ghostbsd.org/node/243) HAMBug online meeting, March 8th @ 18:30 ET (http://hambug.ca/) HardenedBSD 12-STABLE support will be dropped in May 2022 (https://twitter.com/HardenedBSD/status/1492249763193970689) Option options for getopt (https://www.dragonflydigest.com/2022/02/16/26684.html) New Tarsnap version is out (https://mail.tarsnap.com/tarsnap-announce/msg00046.html) pfSense Plus version 22.01 and pfSense CE version 2.6.0 Software are Now Available (https://www.netgate.com/blog/pfsense-plus-software-version-22.01-and-ce-2.6.0-are-now-available) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Karst - replacing disks (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/444/feedback/Karst%20-%20replacing%20disks.md) TheHolm - zfs and booting (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/444/feedback/TheHolm%20-%20zfs%20and%20booting.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 03 Mar 2022 00:00:00 -0800
443: Certified Unix Compliant
Certifying an OS Unix compliant, 2021 FreeBSD Foundation Impact Report, Netflix, Disney, and other widevine content on FreeBSD, file hashes updated for NetBSD 8.1, Playing with CD-RWs on FreeBSD, Why "process substitution" is a late feature in Unix shells, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines What goes into making an OS to be Unix compliant certified? (https://www.quora.com/What-goes-into-making-an-OS-to-be-Unix-compliant-certified) 2021 FreeBSD Foundation Impact Report (https://freebsdfoundation.org/blog/2021-freebsd-foundation-impact-report/) News Roundup Play Netflix, Disney, and other widevine content on FreeBSD (https://danschmid.de/article/play-netflix-disney-and-other-widevine-content-on-freebsd) Note: two files changed and hashes/signatures updated for NetBSD 8.1 (https://bsdsec.net/articles/note-two-files-changed-and-hashes-signatures-updated-for-netbsd-8-1) Playing with CD-RWs on FreeBSD (https://rubenerd.com/playing-with-cd-rws-on-freebsd/) Why "process substitution" is a late feature in Unix shells (https://utcc.utoronto.ca/~cks/space/blog/unix/ProcessSubstitutionWhyLate) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Marty - shell communities (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/443/feedback/Marty%20-%20shell%20communities.md) Nate - Helping Mike Out (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/443/feedback/Nate%20-%20Helping%20Mike%20Out.md) Tom - convincing others to switch (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/443/feedback/Tom%20-%20convincing%20others%20to%20switch.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 24 Feb 2022 00:00:00 -0800
442: Birthing Unix
The Birth of Unix, Help request for three big Lumina items, FreeBSD 13 on Thinkpad T460s, HardenedBSD January 2022 Status Report, OPNsense 22.1 "Observant Owl" released, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines The Birth of Unix (https://klarasystems.com/articles/the-birth-of-unix/) Help requested for three big items for Lumina (https://lumina-desktop.org/post/2022-02-08/) News Roundup FreeBSD 13 on Thinkpad T460s (https://www.tumfatig.net/2022/freebsd-13-on-thinkpad-t460s/) HardenedBSD January 2022 Status Report (https://hardenedbsd.org/article/shawn-webb/2022-01-30/hardenedbsd-january-2022-status-report) OPNsense 22.1 "Observant Owl" released (https://opnsense.org/opnsense-22-1-released/) Beastie Bits The early days of Unix at Bell Labs - Brian Kernighan (LCA 2022 Online) (https://www.youtube.com/watch?v=ECCr_KFl41E) BastilleBSD User Survey (https://docs.google.com/forms/d/e/1FAIpQLSddMMIFW9mHMnpMjMQZfFVCubVywmCXZHI7lqE2tS4k503uPw/viewform) Smallest desktop of the day with BSD: Raspberry Pi 400 (https://www.reddit.com/r/BSD/comments/sgk5y0/smallest_desktop_of_the_day_with_bsd_raspberry_pi/) Reminder BSDCan 2022 - online only (https://lists.bsdcan.org/pipermail/bsdcan-announce/2022-January/000191.html) Joshua Stein Video: Q&A (https://jcs.org/2022/01/14/q&a) DNSSEC Mastery, second edition, creeping out (https://mwl.io/archives/14427) *** ###Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Alec - Playstation FreeBSD-Linux question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/442/feedback/Alec%20-%20Playstation%20FreeBSD-Linux%20question.md) Nelson - Interesting Interview (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/442/feedback/Nelson%20-%20Interesting%20Interview.md) Oscar - Omni OS (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/442/feedback/Oscar%20-%20Omni%20OS.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 17 Feb 2022 00:00:00 -0800
441: Migration to BSD
Migrating our servers from Linux to FreeBSD, Cluster provisioning with Nomad and Pot on FreeBSD, LibBSDDialog, FreeBSD 13.0 Base Jails with ZFS and VNET, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Why we're migrating (many of) our servers from Linux to FreeBSD (https://it-notes.dragas.net/2022/01/24/why-were-migrating-many-of-our-servers-from-linux-to-freebsd/) Cluster provisioning with Nomad and Pot on FreeBSD (https://klarasystems.com/articles/cluster-provisioning-with-nomad-and-pot-on-freebsd/) News Roundup LibBSDDialog (https://alfonsosiciliano.gitlab.io/posts/2022-01-16-libbsddialog.html) FreeBSD 13.0 Base Jails with ZFS and VNET (https://randomnixfix.wordpress.com/2022/01/15/freebsd-13-0-base-jails-with-zfs-and-vnet/) Beastie Bits OpenBSD on the Pinephone (https://www.exoticsilicon.com/crystal/pinephone_openbsd) FreeBSD SSH Hardening (https://gist.github.com/koobs/e01cf8869484a095605404cd0051eb11) Making the ZFS file system (https://changelog.com/podcast/475) A Linux Users Experience Switching To OpenBSD (https://www.youtube.com/watch?v=ukTOfcu1e0w) Add Nix, a purely functional package manager to FreeBSD (https://svnweb.freebsd.org/ports?view=revision&revision=550026) ioztat is a storage load analysis tool for OpenZFS (https://github.com/jimsalterjrs/ioztat) *** ###Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Scott - esxi (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/441/feedback/Scott%20-%20esxi.md) The Holm - noob question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/441/feedback/The%20Holm%20-%20noob%20question.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 10 Feb 2022 00:00:00 -0800
440: BSD Inside Zone
GhostBSD 22.01 is available, Packet Scheduling with Dummynet and FreeBSD, Inside zone installation, Why the FreeBSD Desktop and my Linux Rant, How to install Gnome on OpenBSD, The important Unix idea of the "virtual filesystem switch", and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines GhostBSD 22.01 is available (https://www.ghostbsd.org/ghostbsd_22.01.12_iso_is_now_available) Packet Scheduling with Dummynet and FreeBSD (https://klarasystems.com/articles/packet-scheduling-with-dummynet-and-freebsd/) News Roundup Inside zone installation (https://ptribble.blogspot.com/2022/01/inside-zone-installation.html) Why the FreeBSD Desktop and my Linux Rant (https://randomnixfix.wordpress.com/2021/10/23/why-the-freebsd-desktop-and-my-linux-rant/) How to install Gnome on OpenBSD (https://dataswamp.org/~solene/2021-05-07-openbsd-gnome.html) The important Unix idea of the "virtual filesystem switch" (https://utcc.utoronto.ca/~cks/space/blog/unix/VFSImportance) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Paul - A Plug (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/440/feedback/Paul%20-%20A%20Plug.md) Rollniak - Bhyve Questions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/440/feedback/Rollniak%20-%20Bhyve%20Questions.md) Russell - pf pointers (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/440/feedback/Russell%20-%20pf%20pointers.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 03 Feb 2022 00:00:00 -0800
439: Browser Tab Unix
ACM: It takes a community, Don’t use discord for OSS projects, Unix in a browser tab, OpenIndiana Hipster 2021.10 available, Omni OS CE v11 is out, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines It takes a community - ACM (https://queue.acm.org/detail.cfm?id=3501361) PSA: Dont use Discord for Open Source Projects Jeffrey Paul - Discord Is Not An Acceptable Choice For Free Software Projects (https://sneak.berlin/20200220/discord-is-not-an-acceptable-choice-for-free-software-projects/) Drew deVault - Dont use Discord for FOSS (https://drewdevault.com/2021/12/28/Dont-use-Discord-for-FOSS.html) News Roundup Unix in your Browser Tab (https://browsix.org/) OpenIndiana Hipster 2021.10 is here (https://www.openindiana.org/2021/12/05/openindiana-hipster-2021-10-is-here/) Omni OS CE v11 r151040 is out (https://github.com/omniosorg/omnios-build/blob/r151040/doc/ReleaseNotes.md) Beastie Bits Deb from the FreeBSD Foundation on FLOSS Weekly (https://twit.tv/shows/floss-weekly/episodes/662?autostart=false) Jailfox - BastilleBSD template to bootstrap Firefox. (https://github.com/ddowse/jailfox) FreeBSD Journal Nov/Dec 2021 (https://freebsdfoundation.org/past-issues/storage-2/) First call through the 3ESS (https://www.youtube.com/watch?v=WUUsAK21f20) OpenBSD for minimalists (https://github.com/krzysztofengineer/openbsd) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Dale - two zfs questions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/439/feedback/Dale%20-%20two%20zfs%20questions.md) Johnny - home question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/439/feedback/Johnny%20-%20home%20question.md) Mike - GhostBSD in a VM (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/439/feedback/Mike%20-%20GhostBSD%20in%20a%20VM.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 27 Jan 2022 00:00:00 -0800
438: Toolchain Adventures
FreeBSD Foundation reviews 2021 activities, DragonflyBSD 6.2.1 is here, Lumina Desktop 1.6.2 available, toolchain adventures, The OpenBSD BASED Challenge Day 7, Bastille Template: AdGuard Home, setting up ZSH on FreeBSD and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD Foundation 2021 in Review Software Development (https://freebsdfoundation.org/blog/2021-in-review-software-development/) Year End Fundraising Report (https://freebsdfoundation.org/blog/2021-year-end-fundraising-report/) Infrastructure Support (https://freebsdfoundation.org/blog/2021-in-review-infrastructure-support/) Advocacy (https://freebsdfoundation.org/blog/2021-in-review-advocacy/) FreeBSD 2022 CfP (https://freebsdfoundation.org/blog/freebsd-foundation-2022-call-for-proposals/) DragonFlyBSD 6.2.1 is out (https://www.dragonflybsd.org/release62/) News Roundup Lumina Desktop 1.6.2 is out (https://lumina-desktop.org/post/2021-12-25/) Toolchain Adventures (https://www.cambus.net/toolchains-adventures-q4-2021/) The OpenBSD BASED Challenge Day 7 (https://write.as/adventures-in-bsd/) Bastille Template: AdGuard Home (https://bastillebsd.org/blog/2022/01/03/bastille-template-examples-adguardhome/) Setting up ZSH on FreeBSD (https://www.danschmid.me/article/setting-up-zsh-on-freebsd) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions • Producers Note: We did get some Christmas AMA questions in after we recorded that episode (since we recorded it early) but don't worry, I’ve made a note of them and we’ll save them for our next AMA episode. Patrick - Volume (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/438/feedback/Patrick%20-%20Volume.md) Reptilicus Rex - FreeBSD Docs Team (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/438/feedback/Reptilicus%20Rex%20-%20FreeBSD%20Docs%20Team.md) michael - question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/438/feedback/michael%20-%20question.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 20 Jan 2022 00:00:00 -0800
437: Audit that package
Using FreeBSD’s pkg-audit, 20 year old bug that went to Mars, FreeBSD on Slimbook, LLDB FreeBSD kernel core dump support, Steam on OpenBSD, Cool but obscure X11 tools, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Using FreeBSD’s pkg-audit (https://klarasystems.com/articles/using-freebsds-pkg-audit-to-investigate-known-security-issues/) The 20 year old bug that went to Mars (http://blog.securitymouse.com/2014/06/raising-lazarus-20-year-old-bug-that.html) It's rare that you come across a bug so subtle that it can last for two decades. But, that's exactly what has happened with the Lempel-Ziv-Oberhumer (LZO) algorithm. Initially written in 1994, Markus Oberhumer designed a sophisticated and extremely efficient compression algorithm so elegant and well architected that it outperforms zlib and bzip by four or five times their decompression speed. I was impressed to find out that his LZO algorithm has gone to the planet Mars on NASA devices multiple times! Most recently, LZO has touched down on the red planet within the Mars Curiosity Rover, which just celebrated its first martian anniversary on Tuesday. In the past few years, LZO has gained traction in file systems as well. LZO can be used in the Linux kernel within btrfs, squashfs, jffs2, and ubifs. A recent variant of the algorithm, LZ4, is used for compression in ZFS for Solaris, Illumos, and FreeBSD. With its popularity increasing, Lempel-Ziv-Oberhumer has been rewritten by many engineering firms for both closed and open systems. These rewrites, however, have always been based on Oberhumer's core open source implementation. As a result, they all inherited a subtle integer overflow. Even LZ4 has the same exact bug, but changed very slightly. Because the LZO algorithm is considered a library function, each specific implementation must be evaluated for risk, regardless of whether the algorithm used has been patched. Why? We are talking about code that has existed in the wild for two decades. The scope of this algorithm touches everything from embedded microcontrollers on the Mars Rover, mainframe operating systems, modern day desktops, and mobile phones. Engineers that have used LZO must evaluate the use case to identify whether or not the implementation is vulnerable, and in what format. News Roundup FreeBSD on Slimbook -- 14 months of updates (https://euroquis.nl/freebsd/2021/12/11/slimbook.html) LLDB FreeBSD kernel core dump support (https://www.moritz.systems/blog/lldb-freebsd-kernel-core-dump-support/) Steam on OpenBSD (https://dataswamp.org/~solene/2021-12-01-openbsd-steam.html) Beastie Bits • [OpenSSH Agent Restriction](http://undeadly.org/cgi?action=article;sid=20211220061017) • [OpenBSD’s Clang upgraded to version 13](http://undeadly.org/cgi?action=article;sid=20211220060327) • [Cool, but obscure X11 tools](http://cyber.dabamos.de/unix/x11/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 13 Jan 2022 00:00:00 -0800
436: Unix Standards Battle
UNIX Wars, What every IT person needs to know about OpenBSD Part 3, FreeBSD 12.3 is here, TrueNAS 13 begins, what Unix pre-boot envs looked liked, run Unix on Microcontrollers with PDP-11 emulators and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines UNIX Wars – The Battle for Standards (https://klarasystems.com/articles/unix-wars-the-battle-for-standards/) What every IT person needs to know about OpenBSD Part 3: That packet filter (https://blog.apnic.net/2021/11/11/openbsd-part-3-that-packet-filter/) FreeBSD 12.3-RELEASE Release Notes (https://www.freebsd.org/releases/12.3R/relnotes/) News Roundup TrueNAS 12.0-U7 is Released & TrueNAS 13.0 Begins (https://www.ixsystems.com/blog/truenas-12-0-u7-is-released-truenas-13-0-begins/) A bit on what Unix system pre-boot environments used to look like (https://utcc.utoronto.ca/~cks/space/blog/unix/UnixPreBootEnvironments) RUN UNIX ON MICROCONTROLLERS WITH PDP-11 EMULATOR (https://hackaday.com/2021/11/19/run-unix-on-microcontrollers-with-pdp-11-emulator/) Beastie Bits • [BSDCan 2022 is a go.](https://www.bsdcan.org/2022/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 06 Jan 2022 00:00:00 -0800
435: Year End Interview
In this last episode of 2021, we interview Solene from OpenBSD. She’s blogging about her experiences with OpenBSD on dataswamp.org, the webzine she created, how she got involved and other topics. Enjoy and best wishes for 2022! NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Interview - Solene Rapenne - solene+www@dataswamp.org (mailto:solene+www@dataswamp.org) / @solene@bsd.network (@solene@bsd.network (mastodon)) https://dataswamp.org/~solene/2021-07-26-old-computer-challenge-after.html Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** Special Guest: Solène Rapenne.
Thu, 30 Dec 2021 00:00:00 -0800
434: It’s Quiz-mas time
In this special xmas episode we let the audience interview us using questions they sent us and we’ll answer now. Tom, Allan, JT, and I are all here, so stay tuned for some interesting answers to your questions. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Interview Allan - allanjude@freebsd.org (mailto:allanjude@freebsd.org) / Twitter : @allanjude (https://twitter.com/allanjude) Benedict - bcr@freebsd.org (mailto:bcr@freebsd.org) / Twitter : @bsdbcr (https://twitter.com/bsdbcr) Tom - thj@freebsd.org (mailto:thj@freebsd.org) / Twitter : @adventureloop (https://twitter.com/adventureloop) JT - jt@obs-sec.com (mailto:jt@obs-sec.com) / Twitter : @q5sys (https://twitter.com/q5sys) Tarsnap This week’s episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 23 Dec 2021 00:00:00 -0800
433: GhostBSD of Christmas
GhostBSD 21.11.24 ISO available, why v7 matters so much, OpenBSD on VIA Eden X2 powered HP t510 Thin Client, OctoPkg GUI Package Manager, chdir(2) support in posix_spawn(3), install doas on FreeBSD, Access Modem's Web Interface with OPNsense, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines GhostBSD 21.11.24 ISO is now available (https://www.ghostbsd.org/ghostbsd_21.11.24_iso_is_now_available) Why v7 matters so much (https://utcc.utoronto.ca/~cks/space/blog/unix/V7WhyItMattersSoMuch) News Roundup OpenBSD on the VIA Eden X2 powered HP t510 Thin Client (https://www.cambus.net/openbsd-on-the-via-eden-x2-powered-hp-t510-thin-client/) OctoPkg: A Great GUI Package Manager In FreeBSD (https://nudesystems.com/octopkg-a-great-gui-package-manager-in-freebsd/) Project Report: Add support for chdir(2) support in posix_spawn(3) (https://blog.netbsd.org/tnf/entry/project_report_add_support_for) How To Install doas in FreeBSD 13 (https://nudesystems.com/how-to-install-doas-in-freebsd-13/) How to Access Your Modem's Web Interface with OPNsense (https://homenetworkguy.com/how-to/access-your-modem-web-interface-with-opnsense/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions No feedback for this episode because no one sent any in. :( I guess we’ve answered every BSD and Unix question that everyone has. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 16 Dec 2021 00:00:00 -0800
432: Introducing OpenZFS 3.0 - Yeah
HAMBug hybrid meeting, Demystifying OpenZFS 2.0, OpenZFS 3.0 introduced at Dev Summit, HardenedBSD Home Infrastructure Status, Running Awk in parallel, FreeBSD Announces Wayland 1.19.91, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines HAMBug hybrid meeting (http://hambug.ca/) Hoping to squeeze in an in-person meeting incase the pandemic situation regresses *** ### Demystifying OpenZFS 2.0 (https://klarasystems.com/articles/demystifying-openzfs-2-0/) Do you like the articles we post? We are looking for authors (or even just your ideas) to keep providing these high quality articles. Job Posting (https://lists.freebsd.org/archives/freebsd-jobs/2021-November/000003.html) *** ### OpenZFS 3.0 Introduced at Dev Summit (https://www.ixsystems.com/blog/openzfs-3-0-introduced-at-developer-summit/) *** ### OpenZFS vdev properties feature has been merged (https://github.com/openzfs/zfs/pull/11711) *** News Roundup October 2021 Home Infrastructure Status (https://git.hardenedbsd.org/shawn.webb/articles/-/blob/master/personal/2021-10-20_home_infra/article.md) Running Awk in parallel to process 256M records (https://ketancmaheshwari.github.io/posts/2020/05/24/SMC18-Data-Challenge-4.html) FreeBSD Announce wayland 1.19.91 (https://lists.freedesktop.org/archives/wayland-devel/2021-November/042026.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Brad - running linux binaries under FreeBSD (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/432/feedback/Brad%20-%20running%20linux%20binaries%20under%20FreeBSD.md) Lars - Finding BSD Topics via search engine (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/432/feedback/Lars%20-%20Finding%20BSD%20Topics%20via%20search%20engine.md) Marc - Your views on this question on Reddit (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/432/feedback/Marc%20-%20Your%20views%20on%20this%20question%20on%20Reddit.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 09 Dec 2021 00:00:00 -0800
431: FreeBSD EC2 Agents
Why use OpenBSD part 2, FreeBSD on the RISC-V Architecture, OpenBSD Webzine Issue 4, Ending up liking GNOME, OPNsense 21.7.5 released, Jenkins with FreeBSD Agents in EC2, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines What every IT person needs to know about OpenBSD Part 2: Why use OpenBSD? (https://blog.apnic.net/2021/11/05/openbsd-part-2-why-use-openbsd/) Looking Towards the Future: FreeBSD on the RISC-V Architecture (https://klarasystems.com/articles/looking-towards-the-future-freebsd-on-the-risc-v-architecture/) News Roundup OpenBSD Webzine Issue 4 (https://webzine.puffy.cafe/issue-4.html) How I ended up liking GNOME (https://dataswamp.org/~solene/2021-11-10-how-I-ended-liking-gnome.html) OPNsense 21.7.5 released (https://opnsense.org/opnsense-21-7-5-released/) Jenkins with FreeBSD Agents in ec2 (https://beerdy.io/2021/10/jenkins-with-freebsd-agents-in-ec2/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Andreas - ZFS and Trim (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/431/feedback/Andreas%20-%20ZFS%20and%20Trim.md) Hamza - swift on the BSDs (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/431/feedback/Hamza%20-%20swift%20on%20the%20BSDs.md) Kendall - how many mirror (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/431/feedback/Kendall%20-%20how%20many%20mirrors.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 02 Dec 2021 00:00:00 -0800
430: OpenBSD Onwards
Manipulate a ZFS pool from Rescue System, FreeBSD 3rd Quarter Report, Monitoring FreeBSD jails form the host, OpenBSD on RPI4 with Full Disk Encryption, Onwards with OpenBSD, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Going From Recovery Mode to Normal Operations with OpenZFS Manipulating a Pool from the Rescue System (https://klarasystems.com/articles/manipulating-a-pool-from-the-rescue-system/) Monitoring FreeBSD jails from the host (https://dan.langille.org/2021/10/31/monitoring-freebsd-jails-from-the-host/) News Roundup FreeBSD Quarterly Status Report 3rd Quarter 2021 (https://www.freebsd.org/status/report-2021-07-2021-09/) OpenBSD on Raspberry Pi 4 with Full-Disk Encryption (http://matecha.net/posts/openbsd-on-pi-4-with-full-disk-encryption/) Catchup 2021-11-03 (https://undeadly.org/cgi?action=article;sid=20211103080052) Beastie Bits • [Manage Kubernetes cluster from FreeBSD with kubectl](https://www.youtube.com/watch?v=iUxJIXKtK7c) • [amdgpu support in DragonFly](https://www.dragonflydigest.com/2021/11/08/26343.html) • [Today is the 50th Anniversary of the 1st Edition of Unix...](https://twitter.com/bsdimp/status/1456019089466421248?s=20) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Efraim - response to IPFS and an overlay filesystem (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/430/feedback/Efraim%20-%20response%20to%20IPFS%20and%20an%20overlay%20filesystem.md) Paul - FS Send question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/430/feedback/Paul%20-%20FS%20Send%20question.md) sev - Freebsd & IPA (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/430/feedback/sev%20-%20Freebsd%20%26%20IPA.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 25 Nov 2021 00:00:00 -0800
429: Advanced ZFS Snapshots
FreeBSD Foundation October Fundraising Update, Advanced ZFS Snapshots, Full WireGuard setup with OpenBSD, MidnightBSD a Linux Alternative, FreeBSD Audio, Tuning Power Consumption on FreeBSD Laptops, Thoughts on Spelling Fixes, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD Foundation October 2021 Fundraising Update (https://freebsdfoundation.org/blog/freebsd-foundation-october-2021-fundraising-update/) Advanced ZFS Snapshots (https://klarasystems.com/articles/advanced-zfs-snapshots/) News Roundup Full WireGuard setup with OpenBSD (https://dataswamp.org/~solene/2021-10-09-openbsd-wireguard-exit.html) MidnightBSD a Linux Alternative (https://www.makeuseof.com/midnightbsd-linux-desktop-alternative/) FreeBSD Audio (https://meka.rs/blog/2021/10/12/freebsd-audio/) Tuning Power Consumption on FreeBSD Laptops and Intel Speed Shift (6th Gen and Later) (https://www.neelc.org/posts/freebsd-speed-shift-laptop/) Some Thoughts on Spelling Fixes (http://bsdimp.blogspot.com/2021/10/spelling-fixes-some-advice.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Bens feedback to Benedict's feedback to Bens question about zpoolboy (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/429/feedback/Bens%20feedback%20to%20Benedicts%20feedback%20to%20Bens%20question%20about%20zpoolboy.md) hcddbz - Old Technical Books (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/429/feedback/hcddbz%20-%20Old%20Technical%20Books.md) jason - a jails question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/429/feedback/jason%20-%20a%20jails%20question.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 18 Nov 2021 00:00:00 -0800
428: Cult of BSD
OpenBSD Part 1: How it all started, Explaining top(1) on FreeBSD, Measuring power efficiency of a CPU frequency scheduler on OpenBSD, CultBSD, a whole lot of BSD bits, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines What every IT person needs to know about OpenBSD Part 1: How it all started (https://blog.apnic.net/2021/10/28/openbsd-part-1-how-it-all-started/) Explaining top(1) on FreeBSD (https://klarasystems.com/articles/explaining-top1-on-freebsd/) News Roundup Measuring power efficiency of a CPU frequency scheduler on OpenBSD (https://dataswamp.org/~solene/2021-09-26-openbsd-power-usage.html) CultBSD (https://sourceforge.net/projects/cult-bsd/) Beastie Bits • [OpenBSD on the HiFive Unmatched](https://kernelpanic.life/hardware/hifive-unmatched.html) • [Advanced Documentation Retrieval on FreeBSD](https://adventurist.me/posts/00306) • [OpenBSD Webzine Issue 3 is out](https://webzine.puffy.cafe/issue-3.html) • [How to connect and use Bluetooth headphones on FreeBSD](https://forums.freebsd.org/threads/bluetooth-audio-how-to-connect-and-use-bluetooth-headphones-on-freebsd.82671/) • [How To: Execute Firefox in a jail using iocage and ssh/jailme](https://forums.freebsd.org/threads/how-to-execute-firefox-in-a-jail-using-iocage-and-ssh-jailme.53362/) • [Understanding AWK](https://earthly.dev/blog/awk-examples/) • [“Domesticate Your Badgers” Kickstarter Opens](https://mwl.io/archives/13297) • [Bootstrap an OPNsense development environment in Vagrant](https://github.com/punktDe/vagrant-opnsense) • [VLANs Bridges and LAG Interface best practice questions](https://www.truenas.com/community/threads/vlans-bridges-and-lag-interface-best-practice-questions.93275/) • [A Console Desktop](https://pspodcasting.net/dan/blog/2018/console_desktop.html) • [CharmBUG Casual BSD Meetup and Games (Online)](https://www.meetup.com/CharmBUG/events/281822524) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Dan - ZFS question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/428/feedback/Dan%20-%20ZFS%20question.md) Lars - Thanks for the interview (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/428/feedback/Lars%20-%20Thanks%20for%20the%20interview.md) jesse - migrating data from old laptop (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/428/feedback/jesse%20-%20migrating%20data%20from%20old%20laptop.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 11 Nov 2021 00:00:00 -0800
427: Logging is important
Build Your FreeBSD Developer Workstation, logging is important, how BSD authentication works, pfSense turns 15 years old, OPNsense Business Edition 21.10 released, getting started with pot, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) If you like BSDNow, consider supporting us on Patreon (https://www.patreon.com/bsdnow) Headlines Building Your FreeBSD Developer Workstation Setup (https://klarasystems.com/articles/freebsd-developer-workstation-setup/) What I learned from Russian students: logging is important (https://peter.czanik.hu/posts/russian_students_logging) News Roundup How BSD Authentication works (https://blog.lambda.cx/posts/how-bsd-authentication-works/) pfSense Software is 15 Today! (https://www.netgate.com/blog/pfsense-software-is-15-today) OPNsense® Business Edition 21.10 released (https://opnsense.org/opnsense-business-edition-21-10-released/) Getting started with pot (https://pot.pizzamig.dev/Getting/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. ## Feedback/Questions Benjamin - Question for Benedict (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/427/feedback/Benjamin%20-%20Question%20for%20Benedict.md) Nelson - Episode 419 correction (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/427/feedback/Nelson%20-%20Episode%20419%20correction.md) Peter - state machines (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/427/feedback/Peter%20-%20state%20machines.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 04 Nov 2021 00:00:00 -0700
426: OpenBSD 7.0 Hero
A Good Time to Use OpenZFS Slog, OpenBSD 7.0 is out, OpenBSD and Wayland, UVM faults yield significant performance boost, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines If you like BSDNow, consider supporting us on Patreon (https://www.patreon.com/bsdnow) What Makes a Good Time to Use OpenZFS Slog and When Should You Avoid It (https://klarasystems.com/articles/what-makes-a-good-time-to-use-openzfs-slog-and-when-should-you-avoid-it/) OpenBSD 7.0 is out (https://www.openbsd.org/70.html) News Roundup OpenBSD and Wayland (https://www.sizeofvoid.org/posts/2021-09-26-openbsd-wayland-report/) Unlocking UVM faults yields significant performance boost (https://undeadly.org/cgi?action=article;sid=20210908084117) Beastie Bits PLAN 9 DESKTOP GUIDE (https://pspodcasting.net/dan/blog/2019/plan9_desktop.html) libvirt and DragonFly (https://www.dragonflydigest.com/2021/10/04/26234.html) EuroBSDCon 2021 videos are available (https://undeadly.org/cgi?action=article;sid=20210928192806) Issue#1 of OpenBSD Webzine (https://twitter.com/lcheylus/status/1446553240707993600?s=28) The Beastie has landed. (https://twitter.com/ed_maste/status/1446846780663123968?s=28) It’s 1998 and you are Sun Microsystems... (https://twitter.com/knaversr/status/1443778072113602562) + Reply link that's down (https://web.archive.org/web/20211011003830/https://www.landley.net/history/mirror/unix/srcos.html) RSA/SHA1 signature type disabled by default in OpenSSH (https://undeadly.org/cgi?action=article;sid=20210830113413) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Dan - IPFS (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/426/feedback/Dan%20-%20IPFS.md) Jack - IPFS (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/426/feedback/Jack%20-%20IPFS.md) Johnny - AdvanceBSD (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/426/feedback/Johnny%20-%20AdvanceBSD.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 28 Oct 2021 00:00:00 -0700
425: Releases galore
The New Architecture on the Block, OpenBSD on Vortex86DX CPU, lots of new releases, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines RISC-V: The New Architecture on the Block (https://klarasystems.com/articles/risc-v-the-new-architecture-on-the-block/) If you want more RISC-V, check out JT's interview with Mark Himelstein the CTO of RISC-V International (https://www.opensourcevoices.org/20) *** ### OpenBSD on the Vortex86DX CPU (https://www.cambus.net/openbsd-on-the-vortex86dx-cpu/) *** ## News Roundup aka there’s been lots of releases recently so lets go through them: ### Lumina 1.6.1 (http://lumina-desktop.org/post/2021-10-05/) ### opnsense 21.7.3 (https://opnsense.org/opnsense-21-7-3-released/) ### LibreSSL patches (https://bsdsec.net/articles/openbsd-errata-september-27-2021-libressl) ### OpenBGPD 7.2 (https://marc.info/?l=openbsd-announce&m=163239274430211&w=2) ### Midnight BSD 2.1.0 (https://www.midnightbsd.org/notes/) ### GhostBSD 21.09 ISO (http://ghostbsd.org/ghostbsd_21.09.29_iso_now_available) ### helloSystemv0.6 (https://github.com/helloSystem/ISO/releases/tag/r0.6.0) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Brandon - FreeBSD question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/425/feedback/Brandon%20-%20FreeBSD%20question.md) Bruce - Fixing a weird Apache Bug (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/425/feedback/Bruce%20-%20Fixing%20a%20weird%20Apache%20Bug.md) Dan - zfs question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/425/feedback/Dan%20-%20zfs%20question.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 21 Oct 2021 00:00:00 -0700
424: Unveiling OpenBSD’s pledge
J language working on OpenBSD, Comparing FreeBSD GELI and OpenZFS encrypted pools, What is FreeBSD, actually?, OpenBSD's pledge and unveil from Python, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines I got the J language working on OpenBSD (https://briancallahan.net/blog/20210911.html) Rubenerd: Comparing FreeBSD GELI and OpenZFS encrypted pools with keys (https://rubenerd.com/my-first-prod-encrypted-openzfs-pool/) News Roundup What is FreeBSD, actually? Think again. (https://medium.com/@probonopd/what-is-freebsd-actually-think-again-200c2752d026) OpenBSD's pledge and unveil from Python (https://nullprogram.com/blog/2021/09/15/) Beastie Bits • [Hibernate time reduced](http://undeadly.org/cgi?action=article;sid=20210831050932) • [(open)rsync gains include/exclude support](http://undeadly.org/cgi?action=article;sid=20210830081715) • [Producer JT's latest ancient find that he needs help with](https://twitter.com/q5sys/status/1440105555754848257) • [Doas comes to MidnightBSD](https://github.com/slicer69/doas) • [FreeBSD SSH Hardening](https://gist.github.com/koobs/e01cf8869484a095605404cd0051eb11) • [OpenBSD 6.8 and you](https://home.nuug.no/~peter/openbsd_and_you/#1) • [By default, scp(1) now uses SFTP protocol](https://undeadly.org/cgi?action=article;sid=20210910074941) • [FreeBSD 11.4 end-of-life](https://lists.freebsd.org/pipermail/freebsd-announce/2021-September/002060.html) • [sched_ule(4): Improve long-term load balancer](https://cgit.freebsd.org/src/commit/?id=e745d729be60a47b49eb19c02a6864a747fb2744) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 14 Oct 2021 00:00:00 -0700
423: RACK the Stack
FreeBSD serves Netflix Video at 400Gb/s, Using the RACK TCP stack, an OpenBSD script to update packages fast, Plasma System Monitor and FreeBSD, TrueNAS vs FreeNAS (and why you should upgrade!), auto lock screen on OpenBSD using xidle and xlock, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Serving Netflix Video at 400Gb/s on FreeBSD (https://people.freebsd.org/~gallatin/talks/euro2021.pdf) Using the FreeBSD RACK TCP Stack (https://klarasystems.com/articles/using-the-freebsd-rack-tcp-stack/) News Roundup pkgupdate, an OpenBSD script to update packages fast (https://dataswamp.org/~solene/2021-08-15-openbsd-pkgupdate.html) Plasma System Monitor and FreeBSD (https://euroquis.nl//kde/2021/09/15/systemmonitor.html) TrueNAS vs FreeNAS (and why you should upgrade!) (https://www.ixsystems.com/blog/truenas-vs-freenas-and-why-you-should-upgrade/) Automatically lock screen on OpenBSD using xidle and xlock (https://dataswamp.org/~solene/2021-07-30-openbsd-xidle-xlock.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Ben - LightDM with Slick-Greeter.md (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/423/feedback/Ben%20-%20LightDM%20with%20Slick-Greeter.md) Dave - Cloned Interface.md (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/423/feedback/Dave%20-%20Cloned%20Interface.md) MJ Rodriguez - Sony.md (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/423/feedback/MJ%20Rodriguez%20-%20Sony.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 07 Oct 2021 00:00:00 -0700
422: The Brian Callahan Interview
We interview Dr. Brian Callahan about his language porting work for OpenBSD, teaching with BSDs and recruiting students into projects, research, and his work at NYC*BUG in this week’s episode of BSDnow. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Interview - Dr. Brian Robert Callahan - https://briancallahan.net/ (https://briancallahan.net/) / bcallah@bsdnetwork (https://mastodon.com/bcallah@bsdnetwork) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** Special Guest: Brian Callahan.
Thu, 30 Sep 2021 00:00:00 -0700
421: ZFS eats CPU
Useless use of GNU, Meet the 2021 FreeBSD GSoC Students, historical note on Unix portability, vm86-based venix emulator, ZFS Mysteriously Eating CPU, traceroute gets speed boost, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Useless use of GNU (https://jmmv.dev/2021/08/useless-use-of-gnu.html) Meet the 2021 FreeBSD Google Summer of Code Students (https://freebsdfoundation.org/blog/meet-the-2021-freebsd-google-summer-of-code-students/) News Roundup Large Unix programs were historically not all that portable between Unixes (https://utcc.utoronto.ca/~cks/space/blog/unix/ProgramsVsPortability) References this article: I’m not sure that UNIX won (https://rubenerd.com/im-not-sure-that-unix-won/) *** ### A new path: vm86-based venix emulator (http://bsdimp.blogspot.com/2021/08/a-new-path-vm86-based-venix-emulator.html) *** ### ZFS Is Mysteriously Eating My CPU (http://www.brendangregg.com/blog/2021-09-06/zfs-is-mysteriously-eating-my-cpu.html) *** ### traceroute(8) gets speed boost (http://undeadly.org/cgi?action=article;sid=20210903094704) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Al - TransAtlantic Cables (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/421/feedback/Al%20-%20TransAtlantic%20Cables.md) Christopher - NVMe (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/421/feedback/Christopher%20-%20NVMe.md) JohnnyK - Vivaldi (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/421/feedback/JohnnyK%2-%20Vivaldi.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 23 Sep 2021 00:00:00 -0700
420: OpenBSD makes life better
Choosing The Right ZFS Pool Layout, changes in OpenBSD that make life better, GhostBSD 21.09.06 ISO's now available, Fair Internet bandwidth management with OpenBSD, NetBSD wifi router project update, NetBSD on the Apple M1, HardenedBSD August Status Report, FreeBSD Journal on Wireless and Desktop, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Choosing The Right ZFS Pool Layout (https://klarasystems.com/articles/choosing-the-right-zfs-pool-layout/) Recent and not so recent changes in OpenBSD that make life better (and may turn up elsewhere too) (https://bsdly.blogspot.com/2021/08/recent-and-not-so-recent-changes-in.html) News Roundup GhostBSD 21.09.06 ISO's now available (https://www.ghostbsd.org/ghostbsd_21.09.06_iso_now_available) Fair Internet bandwidth management on a network using OpenBSD (https://dataswamp.org/~solene/2021-08-30-openbsd-qos-lan.html) NetBSD wifi router project update (https://blog.netbsd.org/tnf/entry/wifi_project_status_update) Bonus NetBSD Recent Developments: NetBSD on the Apple M1 (https://mobile.twitter.com/jmcwhatever/status/1431575270436319235) *** ### HardenedBSD August 2021 Status Report (https://hardenedbsd.org/article/shawn-webb/2021-08-31/hardenedbsd-august-2021-status-report) ### FreeBSD Journal July/August 2021: Desktop/Wireless (https://freebsdfoundation.org/past-issues/desktop-wireless/) *** ### Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions James - backup question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/420/feedback/James%20-%20backup%20question.md) Jonathon - certifications (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/420/feedback/Jonathon%20-%20certifications.md) Marty - RPG CLI (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/420/feedback/Marty%20-%20RPG%20CLI.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 16 Sep 2021 00:00:00 -0700
419: Rethinking OS installs
Reviewing a first OpenBSD port, NetBSD 9.2 on a DEC Alpha CPU in QEMU with X11, FreeBSD Experiment Rethinks the OS Install, GhostBSD switching to FreeBSD rc.d, Irix gets LLVM, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Reviewing my first OpenBSD port, and what I'd do differently 10 years later (https://briancallahan.net/blog/20210802.html) Install NetBSD 9.2 on a DEC Alpha CPU in QEMU with X11 (https://raymii.org/s/articles/NetBSD_on_QEMU_Alpha.html) News Roundup FreeBSD Experiment Rethinks the OS Install (https://hackaday.com/2021/08/10/freebsd-experiment-rethinks-the-os-install/) The switch to FreeBSD rc.d is coming (https://www.ghostbsd.org/rc_switch) Irix gets LLVM (https://forums.irixnet.org/thread-3043.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Miceal - a few questions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/419/feedback/Miceal%20-%20a%20few%20questions.md) Nelson - dummynet (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/419/feedback/Nelson%20-%20dummynet.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 09 Sep 2021 00:00:00 -0700
418: The greatest time in history to be a creator
In this episode, we interview Michael W. Lucas about his latest book projects including Git sync murder, TLS Mastery, getting paid for creative work, writing tools and techniques, and more. NOTES Interview - Michael W. Lucas - mwl@mwl.io (mailto:mwl@mwl.io) / @mwlauthor (https://twitter.com/mwlauthor) Cashflow for Creators (https://mwl.io/nonfiction/biz-craft) Charity Auction Against Human Trafficking (https://mwl.io/archives/12526) This is the rfc about what to not do. (https://datatracker.ietf.org/doc/html/rfc9049) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Special Guest: Michael W Lucas.
Thu, 02 Sep 2021 00:45:00 -0700
417: bhyve private cloud
Achieving RPO/RTO Objectives with ZFS pt 1, FreeBSD Foundation Q2 report, OpenBSD full Tor setup, MyBee - bhyve as private cloud, FreeBSD home fileserver expansion, OpenBSD on Framework Laptop, portable GELI, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Achieving RPO/RTO Objectives with ZFS - Part 1 (https://klarasystems.com/articles/achieving-rpo-rto-objectives-with-zfs-part-1/) FreeBSD Foundation Q2 Report (https://freebsdfoundation.org/blog/freebsd-foundation-q2-2021-status-update/) OpenBSD full Tor setup (https://dataswamp.org/~solene/2021-07-25-openbsd-full-tor.html) News Roundup MyBee — FreeBSD OS and hypervisor bhyve as private cloud (https://habr.com/en/post/569226/) Expanding our FreeBSD home file server (https://rubenerd.com/expanding-our-freebsd-home-file-server/) OpenBSD on the Framework Laptop (https://jcs.org/2021/08/06/framework) Portable GELI (http://bijanebrahimi.github.io/blog/portable-geli.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Chunky_pie - zfs question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/417/feedback/Chunky_pie%20-%20zfs%20question.md) Paul - several questions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/417/feedback/Paul%20-%20several%20questions.md) chris - firewall question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/417/feedback/chris%20-%20firewall%20question.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 26 Aug 2021 00:00:00 -0700
416: netcat printing
OpenZFS snapshots, OpenSUSE on Bastille, printing with netcat, new opnsense 21.1.8 released, new pfsense plus software available, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Lets talk OpenZFS snapshots (https://klarasystems.com/articles/lets-talk-openzfs-snapshots/) OpenSUSE in Bastille (https://peter.czanik.hu/posts/opensuse_in_bastille/) News Roundup CUPS printing with netcat (https://retrohacker.substack.com/p/bye-cups-printing-with-netcat) Opnsense-21.1.8 (https://opnsense.org/opnsense-21-1-8-released/) pfSense® Plus Software Version 21.05.1 is Now Available (https://www.netgate.com/blog/pfsense-plus-software-version-21.05.1-is-now-available-for-upgrades) Beastie Bits • [MAC Inspired FreeBSD release](https://github.com/mszoek/airyx) • [Implement unprivileged chroot](https://cgit.freebsd.org/src/commit/?id=a40cf4175c90142442d0c6515f6c83956336699b) • [InitWare: A systemd fork that runs on BSD](https://github.com/InitWare/InitWare) • [multics gets a new release](https://multics-wiki.swenson.org/index.php/Main_Page) • [Open Source Voices interview with Tom Jones](https://www.opensourcevoices.org/17) • [PDP 11/03 Engineering Drawings](https://twitter.com/q5sys/status/1423092689084551171) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Oliver - zfs (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/416/feedback/Olvier%20-%20zfs.md) anders - vms (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/416/feedback/anders%20-%20vms.md) jeff - byhve guests (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/416/feedback/jeff%20-%20byhve%20guests.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 19 Aug 2021 00:00:00 -0700
415: Wrong OS Switch
Wrong Way to Switch Server OS, Net/1 and Net/2 – A Path to Freedom, Permissions Two Mistakes, OpenBSD progress in supporting riscv64 platform, I2P intro, git sync murder is out, GhostBSD init system poll, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines The Wrong Way to Switch Operating Systems on Your Server (https://figbert.com/posts/wrong-way-to-switch-server-os/) History of FreeBSD Part 5: Net/1 and Net/2 – A Path to Freedom (https://klarasystems.com/articles/history-of-freebsd-net-1-and-net-2-a-path-to-freedom/) News Roundup Permissions Two Mistakes (https://utcc.utoronto.ca/~cks/space/blog/unix/PermissionsTwoMistakes) Progress in support for the riscv64 platform (https://undeadly.org/cgi?action=article;sid=20210619161607) I2P Intro (https://dataswamp.org/~solene/2021-06-20-i2p-intro.html) “$ git sync murder” is out, so: how many books have I written? (https://mwl.io/archives/12105) What init system would you prefer to use under GhostBSD? (https://www.ghostbsd.org/what_init_system_pool) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Brad - Replication (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/415/feedback/Brad%20-%20Replication.md) Benedict writes after the show was over: The tool is called https://github.com/allanjude/zxfer Tom tweeted right after recording stopped: https://twitter.com/adventureloop/status/1420478529238622210 Caleb - Pronunciation of Gemini (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/415/feedback/Caleb%20-%20Pronounciation%20of%20Gemini.md) Dan - Writeup about a DO FreeBSD Droplet (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/415/feedback/Dan%20-%20Writeup%20about%20a%20DO%20FreeBSD%20Droplet.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 12 Aug 2021 00:00:00 -0700
414: Running online conferences
OpenZFS 2.1 is out, FreeBSD TCP Performance System Controls, IPFS OpenBSD, tips for running an online conference, fanless OpenBSD laptop, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines OpenZFS 2.1 is out (https://arstechnica.com/gadgets/2021/07/a-deep-dive-into-openzfs-2-1s-new-distributed-raid-topology/) FreeBSD TCP Performance System Controls (https://klarasystems.com/articles/freebsd-tcp-performance-system-controls/) News Roundup IPFS OpenBSD (https://dataswamp.org/~solene/2021-04-17-ipfs-openbsd.html) Tips for running an online conference (https://dan.langille.org/2021/07/23/tips-for-running-an-online-conference/) My Fanless OpenBSD Desktop (https://jcs.org/2021/07/19/desktop) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Bruce - Upgrading (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/414/feedback/Bruce%20-%20Upgrading.md) Chris - SMB Followup (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/414/feedback/Chris%20-%20SMB%20Followup.md) dmilith - kTLS (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/414/feedback/dmilith%20-%20kTLS.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 05 Aug 2021 00:00:00 -0700
413: BSD/Linux Chimera
Updating GCC GNAT (Ada) in pkgsrc/NetBSD, AdvanceBSD thoughts 2/2, FreeBSD from a NetBSD user’s perspective, FPGA programming and DragonFly, Chimera Linux, EuroBSDcon 2021, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Updating GCC GNAT (Ada) in pkgsrc/NetBSD (https://www.irvise.xyz/Projects%20&%20Engineering/updating-gcc-ada-pkgsrc.html) Advance!BSD – thoughts on a not-for-profit project to support *BSD (2/2) (https://eerielinux.wordpress.com/2021/06/20/advancebsd-thoughts-on-a-not-for-profit-project-to-support-bsd-2-2/) News Roundup FreeBSD from a NetBSD user’s perspective (https://washbear.neocities.org/freebsd-netbsd-user.html) FPGA programming and DragonFly (https://mastodon.sdf.org/@yrabbit/106497663837700420) Chimera Linux - A Linux distribution based on FreeBSD userland and LLVM (https://chimera-linux.org/) EuroBSDcon 2021 (https://2021.eurobsdcon.org/about/program/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Charlie - several questions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/413/feedback/Charlie%20-%20several%20questions.md) Dan - kernel driver or module question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/413/feedback/Dan%20-%20kernel%20driver%20or%20module%20question.md) James - Apple M1 (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/413/feedback/James%20-%20Apple%20M1.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 29 Jul 2021 00:00:00 -0700
412: Command-line secrets
FreeBSD Performance Observability, Advance!BSD thoughts 1/2, Lumina Desktop Maintainership Change, How to Handle Secrets on the Command Line, Like NetBSD DragonFlyBSD Now Has "COVID", and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines FreeBSD Performance Observability (https://klarasystems.com/articles/freebsd-performance-observability/) Advance!BSD – thoughts on a not-for-profit project to support *BSD (1/2) (https://eerielinux.wordpress.com/2021/06/20/advancebsd-thoughts-on-a-not-for-profit-project-to-support-bsd-1-2/) News Roundup Maintainership Change :: Lumina Desktop Environment (https://lumina-desktop.org/post/2021-06-23/) Study the past if you would define the Future (https://lumina-desktop.org/post/2021-07-01/) How to Handle Secrets on the Command Line (https://smallstep.com/blog/command-line-secrets/) Following NetBSD, DragonFlyBSD Now Has "COVID" (https://www.phoronix.com/scan.php?page=news_item&px=DragonFlyBSD-COVID) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Jim - freebsd kde (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/412/feedback/Jim%20-%20freebsd%20kde.md) michal - zfs question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/412/feedback/michal%20-%20zfs%20question.md) tim - lumina and snapshots (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/412/feedback/tim%20-%20lumina%20and%20snapshots.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 22 Jul 2021 00:00:00 -0700
411: FreeBSD Deep Dive
Unix System Architecture Evolution, Deep Dive into FreeBSD’s Strengths, how developers chose names, OPNsense 21.1.7 released, Support for chdir(2) in posix_spawn(3), vagrant-freebsd-boxbuilder, OpenBSD’s IATA airport code file, and more This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines The Evolution of the Unix System Architecture (https://www.spinellis.gr/blog/20210618/index.html) • Full IEEE article: https://ieeexplore.ieee.org/document/8704965 Deep Diving Into the Strengths of FreeBSD (https://klarasystems.com/articles/deep-diving-into-the-strengths-of-freebsd/) Interesting read on how Developers choose Names (https://arxiv.org/abs/2103.07487) News Roundup OPNsense 21.1.7 released (https://opnsense.org/opnsense-21-1-7-released/) Support for chdir(2) in posix_spawn(3) (http://blog.netbsd.org/tnf/entry/support_for_chdir_2_in) vagrant-freebsd-boxbuilder (https://github.com/punktDe/vagrant-freebsd-boxbuilder) OpenBSD has a file with 3-letter IATA airport codes (https://twitter.com/jpmens/status/1408825989174546434?s=28) Beastie Bits Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions lyubo - ipfw question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/411/feedback/lyubo%20-%20ipfw%20question.md) michael - a netbsd story (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/411/feedback/michael%20-%20a%20netbsd%20story.md) sven - a dogs garage (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/411/feedback/sven%20-%20a%20dogs%20garage.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 15 Jul 2021 00:00:00 -0700
410: OpenBSD Consumer Gateway
Open Source and Blogging Bubbles, Building Customized FreeBSD Images, Updating Minecraft in FreeBSD, Upgrading FreeBSD jails using mkjail, Dragonfly 6.0 Performance benchmark, OpenBSD Consumer Gateway Launch, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines The Open-Source Software bubble that is and the blogging bubble that was (https://www.baldurbjarnason.com/2021/the-oss-bubble-and-the-blogging-bubble/) Building Customized FreeBSD Images (https://klarasystems.com/articles/building-customized-freebsd-images/) News Roundup Updating to Minecraft 1.17 in FreeBSD (https://rubenerd.com/updating-to-minecraft-1-17-in-freebsd/) Upgrading a FreeBSD 12.2 jail to FreeBSD 13 using mkjail (https://dan.langille.org/2021/05/31/upgrading-a-freebsd-12-2-jail-to-freebsd-13-using-mkjail/) DragonFlyBSD 6.0 Is Performing Very Well Against Ubuntu Linux, FreeBSD 13.0 (https://www.phoronix.com/scan.php?page=article&item=corei9-freebsd13-dfly6&num=1) An OpenBSD Consumer Gateway Launch (https://www.mail-archive.com/misc@openbsd.org/msg178573.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions CY - bearssl (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/410/feedback/CY%20-%20bearssl.md) Marc - that tarsnap ad (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/410/feedback/Marc%20-%20that%20tarsnap%20ad.md) nycbug (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/410/feedback/nycbug.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 08 Jul 2021 00:00:00 -0700
409: The Filesystem Dungeon
DTrace network probes, next 50 years of shell programming, NetBSD on the Vortex86DX CPU, system CPU time in top, your filesystem as a dungeon, diving into toolchains, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines DTrace Network Probes (https://klarasystems.com/articles/dtrace-network-probes/) Unix Shell Programming: The Next 50 Years (https://sigops.org/s/conferences/hotos/2021/papers/hotos21-s06-greenberg.pdf) News Roundup NetBSD on the Vortex86DX CPU (https://www.cambus.net/netbsd-on-the-vortex86dx-cpu/) System CPU time – ‘sys’ time in top (https://blog.ycrash.io/2020/11/28/system-cpu-time-sys-time-in-top/) rpg-cli —your filesystem as a dungeon! (https://github.com/facundoolano/rpg-cli) Diving into toolchains (https://www.cambus.net/diving-into-toolchains/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions • [Alfred - Advice](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/409/feedback/Alfred%20-%20Advice) • [CY - Portable Patch Util](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/409/feedback/CY%20-%20Portable%20Patch%20Util) • [Denis - State of ZFS Ecosystem](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/409/feedback/Denis%20-%20State%20of%20ZFS%20Ecosystem) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 01 Jul 2021 00:00:00 -0700
408: FreeBSD DevSummit 2021
Report from virtual FreeBSD DevSummit 2021, another promising release by FreeBSD Based helloSystem, GearBSD, OpenBGPD release, Let’s Encrypt on OpenBSD, FreeBSD 13 on the Panasonic Let’s Note, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines 2021 FreeBSD Developer Summit (https://klarasystems.com/articles/freebsd-developer-summit-2021/) helloSystem – FreeBSD Based OS Brings another Promising Release 0.5.0 (https://www.debugpoint.com/2021/06/hellosystem-0-5-0-release/) News Roundup GearBSD: a project to help automating your OpenBSD (https://dataswamp.org/~solene/2021-06-01-gearbsd.html) OpenBGPD 7.0 released (https://bsdsec.net/articles/openbgpd-7-0-released) Simple use of Let's Encrypt on OpenBSD is pleasantly straightforward (as of 6.8) (https://utcc.utoronto.ca/~cks/space/blog/unix/OpenBSDNiceLetsEncrypt) FreeBSD 13 on the Panasonic Let’s Note CF-RZ6 (https://rubenerd.com/freebsd-13-on-the-panasonic-cf-rz6/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions • [Paul - ZFS Questions](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/408/feedback/Paul%20-%20ZFS%20Questions) • [Rafael - relic](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/408/feedback/Rafael%20-%20relic) • [matthew - sendfile and ktls](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/408/feedback/matthew%20-%20sendfile%20and%20ktls) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 24 Jun 2021 00:00:00 -0700
407: The jail Detail
Confining the omnipotent root, Jails with ZFS and PF on DigitalOcean, NomadBSD 130R is out, KDE Plasma Wayland on FreeBSD, Firefox under FreeBSD with Privacy, Using NetBSD’s pkgsrc everywhere, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Jails: Confining the omnipotent root (http://phk.freebsd.dk/pubs/sane2000-jail.pdf) A dramatic reading of portions of the paper: Papers We Love: FreeBSD Jails and Solaris Zones (https://paperswelove.org/2016/video/bryan-cantrill-jails-and-solaris-zones/) *** ### Using Jails with ZFS and PF on DigitalOcean (https://medium.com/chris-opperwall/using-jails-with-zfs-and-pf-on-digitalocean-b25b1da82e20) *** ## News Roundup ### NomadBSD 130R is out (https://www.itsfoss.net/nomadbsd-130r-is-now-available-to-download-based-on-freebsd-13-0/) *** ### KDE Plasma Wayland - a week in FreeBSD (https://euroquis.nl//kde/2021/05/09/wayland.html) *** ### Install Firefox under FreeBSD and Set it Up with Privacy (https://danschmid.de/en/blog/install-firefox-under-freebsd-and-set-it-up-with-privacy) *** Using NetBSD’s pkgsrc everywhere I can (https://rubenerd.com/using-netbsds-pkgsrc-everywhere-i-can/) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Malcolm - restoring a single file (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/407/feedback/Malcolm%20-%20restoring%20a%20single%20file) Nathan - wireless support (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/407/feedback/Nathan%20-%20wireless%20support) bluefire - zfs special vdev (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/407/feedback/bluefire%20-%20zfs%20special%20vdev) Push to next show with Allan Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 17 Jun 2021 00:00:00 -0700
406: Jailed Gemini Capsule
Gemini Capsule in a FreeBSD Jail, FreeBSD Quarterly status report 2021Q1, NetBSD VM on bhyve (on TrueNAS), Interview with Michael Lucas, WireGuard Returns as Experimental Package in pfSense, CGI with Awk on OpenBSD httpd, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Gemini Capsule in a FreeBSD Jail (https://www.ecliptik.com/Gemini-Capsule-in-a-FreeBSD-Jail/) With the recent release of FreeBSD 13, I wanted to test it out on a spare RaspberryPi 3 that was part of my old Kubernetes cluster. In particular, FreeBSD Jails have always interested me, although I’ve never used them in practice. Over the years I’ve managed operating system virtualization through Solaris Zones and Docker containers, and Jails seem like and good middle ground between the two - easier to manage than zones and closer to the OS than Docker. I also want to run my own Gemini capsule locally to use some of the features that my other hosted capsules don’t have (like SCGI/CGI) and setting up a capsule in a Jail is a good way to learn both at the same time. FreeBSD Quarterly status report 2021Q1 (https://lists.freebsd.org/pipermail/freebsd-announce/2021-May/002033.html) News Roundup NetBSD VM on bhyve (on TrueNAS) (https://bentsukun.ch/posts/bhyve-netbsd/) My new NAS at home is running TrueNAS Core. So far, it has been excellent, however I struggled a bit setting up a NetBSD VM on it. Part of the problem is that a lot of the docs and how-tos I found are stale, and the information in it no longer applies. TrueNAS Core allows running VMs using bhyve, which is FreeBSD’s hypervisor. NetBSD is not an officially supported OS, at least according to the guest OS chooser in the TrueNAS web UI :) But since the release of NetBSD 9 a while ago, things have become far simpler than they used to be – with one caveat (see below). Interview with Michael Lucas *BSD, Unix, IT and other books author (https://www.cyberciti.biz/interview/michael-lucas-bsd-unix-it-and-other-books-author/) Michael Lucas is a famous IT book author. Perhaps best know for FreeBSD, OpenBSD, and Unix book series. He worked as a system administrator for many years and has now become a full-time book writer. Lately, I did a quick Q and A with Michael about his journey as a professional book author and his daily workflow for writing books. + pfSense – WireGuard Returns as Experimental Package (https://www.netgate.com/blog/pfsense-wireguard-returns-as-an-experimental-package.html) CGI with Awk on OpenBSD httpd (https://box.matto.nl/cgi-with-awk-on-openbsd-httpd.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questionsing Adam - system state during upgrade (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/406/feedback/Adam%20-%20system%20state%20during%20upgrade) paul - BSD grep (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/406/feedback/paul%20-%20BSD%20grep) sub - feedback (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/406/feedback/sub%20-%20feedback) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 10 Jun 2021 00:00:00 -0700
405: OOM Killer Feature
NetBSD 9.2 released, DragonFly 6.0 is out, Home Network Monitoring using Prometheus, Preventing FreeBSD to kill PostgreSQL, Customizing Emacs for Git Commit Messages, Deleting old FreeBSD boot environments, Always be quitting, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines NetBSD 9.2 Released (http://blog.netbsd.org/tnf/entry/netbsd_9_2_released) DragonFly 6.0 is out! (https://www.dragonflydigest.com/2021/05/10/25731.html) Release Notes (https://www.dragonflybsd.org/release60/) *** ### EuroBSDCon 2021 will be online (https://2021.eurobsdcon.org/) *** ## News Roundup ### Home Network Monitoring using Prometheus (https://linux-bsd.github.io/post/monitoring/) > This blog post describes my setup for monitoring various devices on my home network suh as servers, laptops/desktops, networking gear etc. The setup and configuration is squarely geared towards small/medium sized network monitoring. A similar setup might work for large networks, but you will need to plan your compute/storage/bandwidth capacities accordingly. I’m running all the monitoring software on FreeBSD, but you can run it on your choice of OS. Just make sure to install the packages using your OS’s package manager. *** ### Preventing FreeBSD to kill PostgreSQL (aka OOM Killer prevention) (https://fluca1978.github.io/2021/04/02/OOMKillerFreeBSD.html) > There are a lot of interesting articles on how to prevent the Out of Memory Killer (OOM killer in short) on Linux to ruin your day, or better your night. One particularly well done explanation about how the OOM Killer works, and how to help PostgreSQL to survive, is, in my humble opinion, the one from Percona Blog. *** ### Customizing Emacs for Git Commit Messages (http://bsdimp.blogspot.com/2021/04/customizing-emacs-for-git-commit.html) >I do a lot of commits to the FreeBSD project and elsewhere. It would be nice if I could setup emacs in a custom way for each commit message that I'm editing. > Fortunately, GNU Emacs provides a nice way to do just that. While I likely could do some of these things with git commit hooks, I find this to be a little nicer. *** ### Deleting old FreeBSD boot environments (https://dan.langille.org/2021/04/15/deleting-old-freebsd-boot-environments/) > I like boot environments (BE) on FreeBSD. They were especially handy when building the AWS host for FreshPorts, since I had no serial console. I would create a BE saving the current status, then make some changes. I’d mark the current BE as boot once, so I could boot back in the known good BE. Worst case, I could mount the storage onto a rescue EC2 instance and adjust the bootfs value of the zpool. *** Always be quitting (https://jmmv.dev/2021/04/always-be-quitting.html) A good philosophy to live by at work is to “always be quitting”. No, don’t be constantly thinking of leaving your job. But act as if you might leave on short notice. Counterintuitively, this will make you a better engineer and open up growth opportunities. Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Christopher - zfs question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/405/feedback/Christopher%20-%20zfs%20question) Chris - two questions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/405/feedback/Chris%20-%20two%20questions) Vas - zpools and moving to FreeBSD 13 (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/405/feedback/Vas%20-%20zpools%20and%20moving%20to%20FreeBSD%2013) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 03 Jun 2021 00:00:00 -0700
404: 404 BSD Now Hosts Not Found
Allan, Benedict and Tom are MIA, so JT fills in with two friends. This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) CoHosts this week: • Ash Gokhale: https://twitter.com/xpi • Jeff Propes : CoHost of The Opinion Dominion (https://www.theopiniondominion.org) This weeks format follows the format of one of JT's other shows: The Opinion Dominion (https://www.theopiniondominion.org). Centralized vs Decentralized Management Ash’s draid article at Klara (https://klarasystems.com/articles/openzfs-draid-finally/) openbsd’s 50th release (https://twitter.com/openbsd/status/1388289402934333444) + Release Notes (https://www.openbsd.org/69.html) Beastie Bits • Interesting dtrace papers I found this week. The first is unfortunately paywalled by an industry journal but hopefully it’ll be publicly available soon. ◦ [Using Dtrace for Machine Learning Solutions in Malware Detection](https://ieeexplore.ieee.org/document/9225633) ◦ [Process Monitoring on Sequences of System Call Count Vectors](https://arxiv.org/pdf/1707.03821.pdf) ◦ Sounds Similar to: Optimyze Cloud](https://twitter.com/OptimyzeCloud/status/1386424419418099712) CADETS that GNN is working on (https://apps.dtic.mil/sti/citations/AD1080643)] • Practical IOT Hacking book out by no starch press (https://nostarch.com/practical-iot-hacking) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Open Source Voices episode with Colin Percival (https://www.opensourcevoices.org/12) RIP Dan kaminski • https://www.nytimes.com/2021/04/27/technology/daniel-kaminsky-dead.html • https://www.darkreading.com/vulnerabilities---threats/in-appreciation-dan-kaminsky/d/d-id/1340830 • https://www.securityweek.com/security-researcher-dan-kaminsky-passes-away Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 27 May 2021 00:00:00 -0700
403: The Linuxulator Investment
Why You Should Use BSD Licensing for Your Next Open Source Project or Product, Update on FreeBSD Foundation Investment in Linuxulator, OPNsense 21.1.5 released, FreeBSD meetings on the Desktop, Running FreeBSD jails with containerd 1.5, Markdown, DocBook, and the quest for semantic documentation on NetBSD.org, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Why You Should Use BSD Licensing for Your Next Open Source Project or Product (https://klarasystems.com/articles/why-you-should-use-bsd-licensing-for-your-next-open-source-project-or-product/) The term “open source” has its origins in the context of software development, designating a specific approach to developing computer programs. Nowadays, however, it stands for a broad set of values – open source means open exchange, transparency, collaborative participation and development for the benefit of the entire community. Update on FreeBSD Foundation Investment in Linuxulator (https://freebsdfoundation.org/blog/update-on-freebsd-foundation-investment-in-linuxulator/) Dr. Emmett Brown’s similar-sounding Flux Capacitor from the movie Back to the Future bridged the dimension of time, uniting past, present, and future for the McFlys. Similarly, the FreeBSDⓇ Linuxulator project also bridges dimensions – in our case, these are LinuxⓇ and FreeBSD. News Roundup OPNsense 21.1.5 released (https://opnsense.org/opnsense-21-1-5-released/) This is mainly a security and reliablility update. There are several FreeBSD security advisories and updates for third party tools such as curl. + OPNsense to rebase on FreeBSD 13 (https://forum.opnsense.org/index.php?topic=22761.msg108313#msg108313) FreeBSD meetings on the Desktop (https://euroquis.nl//freebsd/2021/04/20/fbsd-bbb.html) FreeBSD on the desktop is a whole stack - X11, Qt, KDE Frameworks, KDE Plasma and KDE Gear, and Wayland, and Poppler and GTK - o my! Running FreeBSD jails with containerd 1.5 (https://samuel.karp.dev/blog/2021/05/running-freebsd-jails-with-containerd-1-5/) containerd 1.5.0 was released today and now works on a new operating system: FreeBSD! This new release includes a series of patches (1, 2, 3, 4, 5, 6, 7, 8, 9, 10) which allow containerd to build, enable the native and zfs snapshotters, and use a compatible runtime like runj. Markdown, DocBook, and the quest for semantic documentation on NetBSD.org (https://washbear.neocities.org/markdown.html) Recently, I’ve been doing a lot of maintenance of the NetBSD website. It contains a boatload of documentation, much of which was originally written in the 2000s. It has some special requirements: it has to work in text-based web browsers like lynx, or maybe even without any working browser installed at all, or just ftp(1) for downloading plain text over HTTP. Naturally, the most important parts are static, suitable for serving from the standard NetBSD http server, which runs from inetd by default. Beastie Bits Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Alrekur - An Interesting FreeBSD Find (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/403/feedback/Alrekur%20-%20An%20Interesting%20FreeBSD%20Find) They presented at the FreeBSD Vendor summit last year too: https://www.youtube.com/watch?v=8LUdZseNrpE Sven - feedback (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/403/feedback/Sven%20-%20feedback) Robert - firewalling (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/403/feedback/Robert%20-%20firewalling) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 20 May 2021 00:00:00 -0700
402: Goodbye GPL
It's time to say goodbye to the GPL, a new OCI Runtime for FreeBSD Jails, A bit of Xenix history, On Updating QEMU's bsd-user fork, FreeBSD 13 on a 12 year old laptop, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines It's time to say goodbye to the GPL (https://martin.kleppmann.com/2021/04/14/goodbye-gpl.html) The trigger for this post is the reinstating of Richard Stallman, a very problematic character, to the board of the Free Software Foundation (FSF). I am appalled by this move, and join others in the call for his removal. This occasion has caused me to reevaluate the position of the FSF in computing. It is the steward of the GNU project (a part of Linux distributions, loosely speaking), and of a family of software licenses centred around the GNU General Public License (GPL). These efforts are unfortunately tainted by Stallman’s behaviour. However, this is not what I actually want to talk about today. runj: a new OCI Runtime for FreeBSD Jails (https://samuel.karp.dev/blog/2021/03/runj-a-new-oci-runtime-for-freebsd-jails/) Today, I open-sourced runj, a new experimental, proof-of-concept OCI-compatible runtime for FreeBSD jails. For the past 6.5 years I’ve been working on Linux containers, but never really had much experience with FreeBSD jails. runj (pronounced “run jay”) is a vehicle for me to learn more about FreeBSD in general and jails in particular. With my position on the Technical Oversight Board of the Open Containers Initiative, I’m also interested in understanding how the OCI runtime specification can be adapted to other operating systems like FreeBSD. News Roundup A Bit of Xenix History (http://seefigure1.com/2014/04/15/xenixtime.html) From 1986 to 1989, I worked in the Xenix1 group at Microsoft. It was my first job out of school, and I was the most junior person on the team. I was hopelessly naive, inexperienced, generally clueless, and borderline incompetent, but my coworkers were kind, supportive and enormously forgiving – just a lovely bunch of folks. On Updating QEMU's bsd-user fork (https://bsdimp.blogspot.com/2021/05/on-updating-qemus-bsd-user-fork.html) FreeBSD 13 on a 12 year old laptop (http://box.matto.nl/freebsd-13-on-a-12-year-old-laptop.html) My old (2009) HP laptop now runs FreeBSD 13.0-RELEASE. Beastie Bits Registration is now open for the June 2021 #FreeBSD Developers Summit (https://twitter.com/i/web/status/1387797859479732227) 6.0RC1 images available (https://www.dragonflydigest.com/2021/04/22/25663.html) Lexical File Names in Plan 9 or Getting Dot-Dot Right (https://plan9.io/sys/doc/lexnames.pdf) The history of UTF-8 as told by Rob Pike (http://doc.cat-v.org/bell_labs/utf-8_history) Initial Support for the riscv64 Architecture (http://undeadly.org/cgi?action=article;sid=20210423090342) *** ###Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Hamza - Congrats on 400 (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/402/feedback/Hamza%20-%20Congrats%20on%20400) Renato - DTS and ContainerD (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/402/feedback/Renato%20-%20DTS%20and%20ContainerD) Rob - Music (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/402/feedback/Rob%20-%20Music) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 13 May 2021 00:00:00 -0700
401: OpenBSD Dog Garage
Dog's Garage Runs OpenBSD, EuroBSDcon 2021 Call for Papers, FreeBSD’s iostat, The state of toolchains in NetBSD, Bandwidth limiting on OpenBSD 6.8, FreeBSD's ports migration to git and its impact on HardenedBSD, TrueNAS 12.0-U3 has been released, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines My Dog's Garage Runs OpenBSD (https://undeadly.org/cgi?action=article;sid=20210415055717) I was inspired by the April 2017 article in undeadly.org about getting OpenBSD running on a Raspberry Pi 3B+. My goal was to use a Raspberry Pi running OpenBSD to monitor the temperature in my garage from my home. My dog has his own little "apartment" inside the garage, so I want to keep an eye on the temperature. (I don't rely on this device. He sleeps inside the house whenever he wants.) EuroBSDcon 2021 Call for Papers (https://2021.eurobsdcon.org/about/cfp/) FreeBSD iostat (https://klarasystems.com/articles/freebsd-iostat-a-quick-glance/) The state of toolchains in NetBSD (https://www.cambus.net/the-state-of-toolchains-in-netbsd/) While FreeBSD and OpenBSD both switched to using LLVM/Clang as their base system compiler, NetBSD picked a different path and remained with GCC and binutils regardless of the license change to GPLv3. However, it doesn't mean that the NetBSD project endorses this license, and the NetBSD Foundation's has issued a statement about its position on the subject. NetBSD’s statement (http://cvsweb.netbsd.org/bsdweb.cgi/src/external/gpl3/README?rev=1.1) *** News Roundup Bandwidth limiting on OpenBSD 6.8 (https://dataswamp.org/~solene/2021-02-07-limit.html) I will explain how to limit bandwidth on OpenBSD using its firewall PF (Packet Filter) queuing capability. It is a very powerful feature but it may be hard to understand at first. What is very important to understand is that it's technically not possible to limit the bandwidth of the whole system, because once data is getting on your network interface, it's already there and got by your router, what is possible is to limit the upload rate to cap the download rate. FreeBSD's ports migration to git and its impact on HardenedBSD (https://hardenedbsd.org/article/shawn-webb/2021-04-06/freebsds-ports-migration-git-and-its-impact-hardenedbsd) FreeBSD completed their ports migration from subversion to git. Prior to the official switch, we used the read-only mirror FreeBSD had at GitHub[1]. The new repo is at [2]. A cursory glance at the new repo will show that the commit hashes changed. This presents an issue with HardenedBSD's ports tree in our merge-based workflow. TrueNAS 12.0-U3 has been released (https://www.truenas.com/docs/releasenotes/core/12.0u3/) iXsystems is excited to announce TrueNAS 12.0-U3 was released today and marks an important milestone in the transition from FreeNAS to TrueNAS. TrueNAS 12.0 is now considered by iXsystems to be a higher quality release than FreeNAS 11.3-U5, our previous benchmark. The new TrueNAS documentation site has also reached a point where it has more content and capabilities than FreeNAS. TrueNAS 12.0 is ready for mission-critical enterprise deployments. Beastie Bits Joyent provides pkgsrc for MacOS X (https://pkgsrc.joyent.com/install-on-osx/) Archives of old Irix documentation (https://techpubs.jurassic.nl) FreeBSD Developer/Vendor Summit 2021 (https://wiki.freebsd.org/DevSummit/202106) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Andre - splitting zfs array (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/401/feedback/Andre - splitting zfs array) Bruce - Command Change (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/401/feedback/Bruce - Command Change) Dan - Annoyances with ZFS (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/401/feedback/Dan - Annoyances with ZFS) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 06 May 2021 00:00:00 -0700
400: FreeBSD became 13
FreeBSD 13 is here, multi-factor authentication on OpenBSD, KDE on FreeBSD 2021o2, NetBSD GSoC report, a working D compiler on OpenBSD, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines FreeBSD 13.0 R Annoucement (https://www.freebsd.org/releases/13.0R/announce/) • OpenZFS 2.0 (almost 2.1) is included in 13.0 • Removed support for previously-deprecated algorithms in geli(8). • The armv8crypto(4) driver now supports AES-GCM which is used by IPsec and kernel TLS. Enable multi-factor authentication on OpenBSD (https://dataswamp.org/~solene/2021-02-06-openbsd-2fa.html) In this article I will explain how to add a bit more security to your OpenBSD system by adding a requirement for user logging into the system, locally or by ssh. I will explain how to setup 2 factor authentication (2FA) using TOTP on OpenBSD News Roundup KDE on FreeBSD 2021o2 (https://euroquis.nl/kde/2021/03/26/freebsd2021o2.html) Gosh, second octant already! Well, let’s take a look at the big things that happened in KDE-on-FreeBSD in these six-and-a-half weeks. GSoC Reports: Make system(3), popen(3) and popenve(3) use posix_spawn(3) internally (Final report) (http://blog.netbsd.org/tnf/entry/gsoc_reports_make_system_31) My code can be found at github.com/teknokatze/src in the gsoc2020 branch, at the time of writing some of it is still missing. The test facilities and logs can be found in github.com/teknokatze/gsoc2020. A diff can be found at github which will later be split into several patches before it is sent to QA for merging. The initial and defined goal of this project was to make system(3) and popen(3) use posixspawn(3) internally, which had been completed in June. For the second part I was given the task to replace fork+exec calls in our standard shell (sh) in one scenario. Similar to the previous goal we determined through implementation if the initial motivation, to get performance improvements, is correct otherwise we collect metrics for why posixspawn() in this case should be avoided. This second part meant in practice that I had to add and change code in the kernel, add a new public libc function, and understand shell internals. A working D compiler on OpenBSD (https://undeadly.org/cgi?action=article;sid=20210322080633) Dr. Brian Robert Callahan (bcallah@) blogged about his work in getting D compiler(s) working under OpenBSD. + Full Post (https://briancallahan.net/blog/20210320.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Vasilis - upgrade question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/400/feedback/Vasilis%20-%20upgrade%20question) Dennis - zfs questions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/400/feedback/Dennis%20-%20zfs%20questions) Daniel Dettlaff - KTLS question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/400/feedback/dmilith%20-%20KTLS) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 29 Apr 2021 00:00:00 -0700
399: Comparing Sandboxes
Comparing sandboxing techniques, Statement on FreeBSD development processes, customizing FreeBSD ports and packages, the quest for a comfortable NetBSD desktop, Nginx as a TCP/UDP relay, HardenedBSD March 2021 Status Report, Detailed Behaviors of Unix Signal, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Comparing sandboxing techniques (https://www.omarpolo.com/post/gmid-sandbox.html) I had the opportunity to implement a sandbox and I'd like to write about the differences between the various sandboxing techniques available on three different operating systems: FreeBSD, Linux and OpenBSD. Statement on FreeBSD development processes (https://lists.freebsd.org/pipermail/freebsd-hackers/2021-March/057127.html) In light of the recent commentary on FreeBSD's development practices, members of the Core team would like to issue the following statement. Customizing FreeBSD Ports and Packages (https://klarasystems.com/articles/customizing-freebsd-ports-and-packages/) A basic intro to building your own packages News Roundup FVWM(3) and the quest for a comfortable NetBSD desktop (https://www.unitedbsd.com/d/442-fvwm3-and-the-quest-for-a-comfortable-netbsd-desktop) FVWM substantially allows one to build a fully-fledged lightweight desktop environment from scratch, with an almost unparalleled degree of freedom. Although using FVWM does not require any knowledge of programming languages, it is possible to extend it with M4, C, and Perl preprocessing. Nginx as a TCP/UDP relay (https://dataswamp.org/~solene/2021-02-24-nginx-stream.html) In this tutorial I will explain how to use Nginx as a TCP or UDP relay as an alternative to Haproxy or Relayd. This mean nginx will be able to accept requests on a port (TCP/UDP) and relay it to another backend without knowing about the content. It also permits to negociates a TLS session with the client and relay to a non-TLS backend. In this example I will explain how to configure Nginx to accept TLS requests to transmit it to my Gemini server Vger, Gemini protocol has TLS as a requirement. HardenedBSD March 2021 Status Report (https://hardenedbsd.org/article/shawn-webb/2021-03-31/hardenedbsd-march-2021-status-report) This month, I worked on finding and fixing the regression that caused kernel panics on our package builders. I think I found the issue: I made it so that the HARDENEDBSD amd64 kernel just included GENERIC so that we follow FreeBSD's toggling of features. Doing so added QUEUEMACRODEBUGTRASH to our kernel config. That option is the likely culprit. If the next package build (with the option removed) completes, I will commit the change that removes QUEUEMACRODEBUGTRASH from the HARDENEDBSD amd64 kernel. Detailed Behaviors of Unix Signal (https://www.dyx.name/posts/essays/signal.html) When Unix is mentioned in this document it means macOS or Linux as they are the mainly used Unix at this moment. When shell is mentioned it means Bash or Zsh. Most demos are written in C for macOS with Apple libc and Linux with glibc. Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions andrew - flatpak (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/399/feedback/andrew%20-%20flatpak) chris - mac and truenas (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/399/feedback/chris%20-%20mac%20and%20truenas) robert - some questions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/399/feedback/robert%20-%20some%20questions) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 22 Apr 2021 00:00:00 -0700
398: Coordinated Mars Time
FreeBSD 13.0 Full Desktop Experience, FreeBSD on ARM64 in the Cloud, Plan 9 from Bell Labs in Cyberspace, Inferno is open source as well, NetBSD hits donation milestone, grep returns (standard input) on FreeBSD, Random Programming Challenge, OpenBSD Adds Support for Coordinated Mars Time (MTC) and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines FreeBSD 13.0 – Full Desktop Experience (https://www.tubsta.com/2021/03/freebsd-13-0-full-desktop-experience/) With the release of FreeBSD 13.0 on the horizon, I wanted to see how it shapes up on my Lenovo T450 laptop. Previous major releases on this laptop, using it as a workstation, felt very rough around the edges but with 13, it feels like the developers got it right. FreeBSD on ARM64 in the Cloud (https://klarasystems.com/articles/the-next-level-freebsd-on-arm64-in-the-cloud/) Until the end of June, Amazon AWS is offering free ARM64 Graviton instances, learn how to try out FreeBSD to ARMv8 in the cloud Plan 9 from Bell Labs in Cyberspace! (https://www.bell-labs.com/institute/blog/plan-9-bell-labs-cyberspace/) The releases below represent the historical releases of Plan 9. (http://p9f.org/dl/index.html) The two versions of 4th Edition represent the initial release and the final version available from Bell Labs as it was updated and patched. All historical releases of Plan 9 have been re-released under the terms of the MIT license. + Inferno is open source as well (https://bitbucket.org/inferno-os/inferno64-os/src/master/) News Roundup Hitting donation milestone, financial report for 2020 (http://blog.netbsd.org/tnf/entry/hitting_donation_milestone_financial_report) We nearly hit our 2020 donation milestone set after the release of 9.0 of $50,000. grep returns (standard input) on FreeBSD (https://rubenerd.com/grep-returns-standard-input/) I was dealing with a bizarre error with grep(1) on FreeBSD, and it soon infected my macOS and NetBSD machines too. It was driving me crazy! Random Programming Challenge (https://projecteuler.net/problem=84) This better not be an April Fools Joke… I want to see this actually implemented. I’ll donate $100 to the first BSD that actually implements this for real. Who’s with me? OpenBSD Adds Support for Coordinated Mars Time (MTC) (https://marc.info/?l=openbsd-cvs&m=161730046519995) To make sure that OpenBSD can be used elsewhere than just earth, this diff introduces Coordinated Mars Time (MTC), the Mars equivalent of earth’s Universal Time (UTC). OpenZFS had a good one too (https://github.com/openzfs/zfs/pull/11823) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Brandon - router (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/398/feedback/Brandon%20-%20router) Lawrence - Is BSD for me (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/398/feedback/Lawrence%20-%20Is%20FreeBSD%20for%20me) miguel - printing (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/398/feedback/miguel%20-%20printing) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 15 Apr 2021 00:00:00 -0700
397: Fresh BSD 2021
Customizing the FreeBSD Kernel, OpenBSD/loongson on the Lemote Fuloong, how ZFS on Linux brings up pools and filesystems at boot under systemd, LLDB: FreeBSD Legacy Process Plugin Removed, FreshBSD 2021, gmid, Danschmid’s Poudriere Guide in english, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Customizing the FreeBSD Kernel (https://klarasystems.com/articles/customizing-the-freebsd-kernel/) Learn more about customizing the build of the FreeBSD kernel and its loadable modules OpenBSD/loongson on the Lemote Fuloong (https://www.cambus.net/openbsd-loongson-on-the-lemote-fuloong/) In my article about running OpenBSD/loongson on the Lemote Yeeloong back in 2016, I mentioned looking for a Fuloong. All hope seemed lost until the Summer of 2017, when a fellow OpenBSD developer was contacted by a generous user (Thanks again, Lars!) offering to donate two Lemote Fuloong machines, and I was lucky enough to get one of those units. News Roundup How ZFS on Linux brings up pools and filesystems at boot under systemd (https://utcc.utoronto.ca/~cks/space/blog/linux/ZFSBringUpOnBoot) On Solaris and Illumos, how ZFS pools and filesystems were brought up at boot was always a partial mystery to me (and it seemed to involve the kernel knowing a lot about /etc/zfs/zpool.cache). On Linux, additional software RAID arrays are brought up mostly through udev rules, which has its own complications. For a long time I had the general impression that ZFS on Linux also worked through udev rules to recognize vdev components, much like software RAID. However, this turns out to not be the case and the modern ZFS on Linux boot process is quite straightforward on systemd systems. LLDB: FreeBSD Legacy Process Plugin Removed (https://www.moritz.systems/blog/freebsd-legacy-process-plugin-removed/) During the past month we’ve successfully removed the legacy FreeBSD plugin and continued improving the new one. We have prepared an implementation of hardware breakpoint and watchpoint support for FreeBSD/AArch64, and iterated over all tests that currently fail on that platform. Therefore, we have concluded the second milestone. FreshBSD 2021 (https://freshbsd.org/news/2021/02/28) 6 weeks ago I created a branch for a significant rework of FreshBSD. Nearly 300 commits later, and just a week shy of our 15th anniversary, the result is what you’re looking at now. I hope you like it. gmid (https://github.com/omar-polo/gmid/) is a gemini (https://gemini.circumlunar.space/) server for unixes. Danschmid’s Poudriere Guide now in english (https://danschmid.de/en/blog/poudriere-guide) The ports system is one of FreeBSD's greatest advantages for users who want flexibility and control over their software. It enables administrators to easily create and manage source-based installations using a system that is robust and predictable. Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** Special Guest: Tom Jones.
Thu, 08 Apr 2021 00:00:00 -0700
396: License to thrill
FreeBSD Network Troubleshooting, The State of FreeBSD, dhcpleased, bhyve for Calamares Development, EFS automount and ebsnvme-id, Old Usenix pictures, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines FreeBSD Network Troubleshooting (https://klarasystems.com/articles/freebsd-network-troubleshooting-understanding-network-performance/) FreeBSD has a full set of debugging features, and the network stack is able to report a ton of information. So much that it can be hard to figure out what is relevant and what is not. The State of FreeBSD (https://www.theregister.com/2021/03/10/the_state_of_freebsd/) License to thrill: Ahead of v13.0, the FreeBSD team talks about Linux and the completed toolchain project that changes everything News Roundup dhcpleased(8) - DHCP client daemon (http://undeadly.org/cgi?action=article;sid=20210227232424) With the following commit, Florian Obser (florian@) imported dhcpleased(8), DHCP daemon to acquire IPv4 address leases from servers, plus dhcpleasectl(8), a utility to control the daemon: bhyve for Calamares Development (https://euroquis.nl//freebsd/2021/03/05/bhyve.html) bhyve (pronounced “bee hive”) is a hypervisor for BSD systems (and Illumos / openSolaris). It is geared towards server workloads, but does support desktop-oriented operation as well. I spent some time wayyyy back in November wrestling with it in order to replace VirtualBox for Calamares testing on FreeBSD. The “golden hint” as far as I’m concerned came from Karen Bruner and now I have a functioning Calamares test-ground that is more useful than before. “Calamares is a free and open-source independent and distro-agnostic system installer for Linux distributions.“ Some new FreeBSD/EC2 features: EFS automount and ebsnvme-id (https://www.daemonology.net/blog/2020-05-31-Some-new-FreeBSD-EC2-features.html) As my regular readers will be aware, I've been working on and gradually improving FreeBSD/EC2 for many years. Recently I've added two new features, which are available in the weekly HEAD and 12-STABLE snapshots and will appear in releases starting from 12.2-RELEASE. Old Usenix pictures (http://lists.nycbug.org/pipermail/talk/2021-February/018304.html) Beastie Bits https://2021.eurobsdcon.org/ (CFP is open until May 26th, 2021) EuroBSDcon is the European technical conference for users and developers of BSD-based systems. The conference is scheduled to take place September 16-19 2021 in Vienna, Austria or as an all-online event if COVID-19 developments dictate. The tutorials will be held on Thursday and Friday to registered participants and the talks are presented to conference attendees on Saturday and Sunday. The Call for Talk and Presentation proposals period will close on May 26th, 2021. Prospective speakers will be notified of acceptance or otherwise by June 1st, 2021. https://campgnd.com/ (CFP is open until 2021-04-15) campgndd will be held May 28th, 29th and 30th 2021, from wherever you happen to be. We're looking for submissions on anything you're enthusiastic and excited about. If you enjoy it, the odds are we will too! You don't need to be an expert to propose anything. Some example of things we are looking for are: Talks Walkthroughs Music From the Desk of Michael Lucas… ``` New Release: Only Footnotes I’ve lost count of the number of people who have told me that they purchase my books only for the footnotes. That’s okay. I don’t care why people buy my books, only that they do buy them. Nevertheless, I am a businessman living under capitalism and feel compelled to respond to my market. Allow me to present my latest release: Only Footnotes, a handsome hardcover-only compilation of decades of footnotes. From the back cover: Only Footnotes. Because that’s why you read his books. Academics hate footnotes. Michael W Lucas loves them. What he does with them wouldn’t pass academic muster, but that doesn’t mean the reader should skip them. The footnotes are the best part! Why not read only the footnotes, and skip all that other junk? After literal minutes of effort, Only Footnotes collects every single footnote from all of Lucas’ books to date.* Recycle those cumbersome treatises stuffed with irrelevant facts! No more flipping through pages and pages of actual technical knowledge looking for the offhand movie reference or half-formed joke. This slender, elegant volume contains everything the man ever passed off as his dubious, malformed “wisdom.” Smart books have footnotes. Smarter books are only footnotes. *plus additional annotations from the author. Because sometimes even a footnote needs a footnote. With interior illustrations by OpenBSD’s akoshibe, this distinguished tome would make fine inspirational reading for a system administrator, network engineer, or anyone sentenced to a life in information technology. Available at all fine bookstores, and many mediocre ones! ``` Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** Special Guest: Tom Jones.
Thu, 01 Apr 2021 00:00:00 -0700
395: Tracing ARM’s history
Tracing the History of ARM and FreeBSD, Make ‘less’ more friendly, NomadBSD 1.4 Release, Create an Ubuntu Linux jail on FreeBSD 12.2, OPNsense 21.1.2 released, Midnight BSD and BastilleBSD, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Tracing the History of ARM and FreeBSD (https://klarasystems.com/articles/tracing-the-history-of-arm-and-freebsd/) When we think of computers, we generally think of laptops and desktops. Each one of these systems is powered by an Intel or AMD chip based on the x86 architecture. It might feel like you spend all day interacting with these kinds of systems, but you would be wrong. Unix Tip: Make ‘less’ more friendly (https://ascending.wordpress.com/2011/02/11/unix-tip-make-less-more-friendly/) You probably know about less: it is a standard tool that allows scrolling up and down in documents that do not fit on a single screen. Less has a very handy feature, which can be turned on by invoking it with the -i flag. This causes less to ignore case when searching. For example, ‘udf’ will find ‘udf’, ‘UDF’, ‘UdF’, and any other combination of upper-case and lower-case. If you’re used to searching in a web browser, this is probably what you want. But less is even more clever than that. If your search pattern contains upper-case letters, the ignore-case feature will be disabled. So if you’re looking for ‘QXml’, you will not be bothered by matches for the lower-case ‘qxml’. (This is equivalent to ignorecase + smartcase in vim.) News Roundup NomadBSD 1.4 Release (https://www.itsfoss.net/nomadbsd-1-4-release/) Version 1.4 of NomadBSD, a persistent live system for USB flash drives based on FreeBSD and featuring a graphical user interface built around Openbox, has been released: “We are pleased to present the release of NomadBSD 1.4. Create an Ubuntu Linux jail on FreeBSD 12.2 (https://hackacad.net/post/2021-01-23-create-a-ubuntu-linux-jail-on-freebsd/) OPNsense 21.1.2 released (https://opnsense.org/opnsense-21-1-2-released/) Work has so far been focused on the firmware update process to ensure its safety around edge cases and recovery methods for the worst case. To that end 21.1.3 will likely receive the full revamp including API and GUI changes for a swift transition after thorough testing of the changes now available in the development package of this release. Midnight BSD and BastilleBSD (https://www.justjournal.com/users/mbsd/entry/33869) We recently added a new port, mports/sysutils/bastille that allows you to manage containers. This is a port of a project that originally targetted FreeBSD, but also works on HardenedBSD. Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Brad - monitoring with Grafana (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/395/feedback/Brad%20-%20monitoring%20with%20Grafana) Dennis - a few questions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/395/feedback/Dennis%20-%20a%20few%20questions) Paul - FreeBSD 13 (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/395/feedback/Paul%20-%20FreeBSD%2013) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 25 Mar 2021 00:00:00 -0700
394: FreeBSD on Mars
Onboard Scheduler for the Mars 2020 Rover, Practical Guide to Storage of Large Amounts of Microscopy Data, OpenBSD guest with bhyve - OmniOS, NextCloud on OpenBSD, MySQL Transactions - the physical side, TrueNAS 12.0-U2.1 is released, HardenedBSD 2021 State of the Hardened Union, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Prototyping an Onboard Scheduler for the Mars 2020 Rover (https://ai.jpl.nasa.gov/public/documents/papers/rabideau_iwpss2017_prototyping.pdf) The mars rover runs VxWorks, which is based on BSD, and uses the FreeBSD networking stack. While there has been a lot of type about the little helicopter that was inside the rover running Linux, the rover itself runs BSD. *** ### Practical Guide to Storage of Large Amounts of Microscopy Data (https://www.cambridge.org/core/journals/microscopy-today/article/practical-guide-to-storage-of-large-amounts-of-microscopy-data/D3CE39447BFF5BBF9B3ED8A0C35C6F36) > Biological imaging tools continue to increase in speed, scale, and resolution, often resulting in the collection of gigabytes or even terabytes of data in a single experiment. In comparison, the ability of research laboratories to store and manage this data is lagging greatly. This leads to limits on the collection of valuable data and slows data analysis and research progress. Here we review common ways researchers store data and outline the drawbacks and benefits of each method. We also offer a blueprint and budget estimation for a currently deployed data server used to store large datasets from zebrafish brain activity experiments using light-sheet microscopy. Data storage strategy should be carefully considered and different options compared when designing imaging experiments. *** ## News Roundup ### OpenBSD guest with bhyve - OmniOS (https://www.pbdigital.org/omniosce/bhyve/openbsd/2020/06/08/bhyve-zones-omnios.html) > Today I will be creating a OpenBSD guest via bhyve on OmniOS. I will also be adding a Pass Through Ethernet Controller so I can have a multi-homed guest that will serve as a firewall/router. > This post will cover setting up bhyve on OmniOS, so it will also be a good introduction to bhyve. As well, I look into OpenBSD’s uEFI boot loader so if you have had trouble with this, then you are in the right place. *** ### NextCloud on OpenBSD (https://h3artbl33d.nl/blog/nextcloud-on-openbsd) > NextCloud and OpenBSD are complimentary to one another. NextCloud is an awesome, secure and private alternative for propietary platforms, whereas OpenBSD forms the most secure and solid foundation to serve it on. Setting it up in the best way isn’t hard, especially using this step by step tutorial. MySQL Transactions - the physical side (https://blog.koehntopp.info/2020/07/27/mysql-transactions.html) So you talk to a database, doing transactions. What happens actually, behind the scenes? Let’s have a look. TrueNAS 12.0-U2.1 is released (https://www.truenas.com/docs/hub/intro/release-notes/12.0u2.1/) HardenedBSD 2021 State of the Hardened Union - NYCBUG - 2021-04-07 (https://www.nycbug.org/index?action=view&id=10682) Beastie Bits FreeBSD Journal: Case Studies (https://freebsdfoundation.org/our-work/journal/) *** ###Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Al - BusyNAS (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/394/feedback/Al%20-%20BusyNAS) Jeff - ZFS and NFS on FreeBSD (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/394/feedback/Jeff%20-%20ZFS%20and%20NFS%20on%20FreeBSD) Michael - remote unlock for encrypted systems (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/394/feedback/Michael%20-%20remote%20unlock%20for%20encrypted%20systems) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 18 Mar 2021 00:00:00 -0700
393: ZFS dRAID
Lessons learned from a 27 years old UNIX book, Finally dRAID, Setting up a Signal Proxy using FreeBSD, Annotate your PDF files on OpenBSD, Things You Should Do Now, Just: More unixy than Make, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Lessons learned from a 27 years old UNIX book (https://www.linux.it/~ema/posts/porsche-book/) One of the Amazon reviewers of "Sun Performance and Tuning: Java and the Internet" gave it 3/5 stars. While still a nice introduction, the book by Adrian Cockcroft has become dated — claimed Roland in 2003, which believe it or not was 18 years ago... dRAID, Finally! (https://klarasystems.com/articles/openzfs-draid-finally/) Admins will often use wide RAID stripes to maximize usable storage given a number of spindles. RAID-Z deployments with large stripe widths, ten or larger, are subject to poor resilver performance for a number of reasons. Resilvering a full vdev means reading from every healthy disk and continuously writing to the new spare. This will saturate the replacement disk with writes while scattering seeks over the rest of the vdev. For 14 wide RAID-Z2 vdevs using 12TB spindles, rebuilds can take weeks. Resilver I/O activity is deprioritized when the system has not been idle for a minimum period. Full zpools get fragmented and require additional I/O’s to recalculate data during reslivering. A pool can degenerate into a never ending cycle of rebuilds or loss of the pool Aka: the Death Spiral. News Roundup Setting up a Signal Proxy using FreeBSD (https://www.neelc.org/posts/freebsd-signal-proxy/) With the events that the private messaging app Signal has been blocked in Iran, Signal has come up with an “proxy” solution akin to Tor’s Bridges, and have given instructions on how to do it. For people who prefer FreeBSD over Linux like myself, we obviously can’t run Docker, which is what Signal’s instructions focus on. Fortunately, the Docker image is just a fancy wrapper around nginx, and the configs can be ported to any OS. Here, I’ll show you how to set up a Signal Proxy on FreeBSD. Annotate your PDF files on OpenBSD (https://www.tumfatig.net/20210126/annotate-your-pdf-files-on-openbsd) On my journey to leave macOS, I regularly look to mimic some of the features I use. Namely, annotating (or signing) PDF files is a really simple task using Preview. I couldn’t do it on OpenBSD using Zathura, Xpdf etc. But there is a software in the ports that can achieve this: Xournal. Xournal is “an application for notetaking, sketching, keeping a journal using a stylus“. And now that my touchscreen is calibrated, highlighting can even be done with the fingers :) Things You Should Do Now (https://secure.phabricator.com/book/phabflavor/article/things_you_should_do_now/) Describes things you should do now when building software, because the cost to do them increases over time and eventually becomes prohibitive or impossible. Just: A command runner. More unixy than Make because it does even less. (https://github.com/casey/just/) I think it's in the do-one-thing-well spirit of Unix, because it's just a command runner, no build system at all. Just has a bunch of nice features: Can be invoked from any subdirectory Arguments can be passed from the command line Static error checking that catches syntax errors and typos Excellent error messages with source context The ability to list recipes from the command line Recipes can be written in any language Works on Linux, macOS, and Windows And much more! Just doesn't replace Make, or any other build system, but it does replace reverse-searching your command history, telling colleagues the weird flags they need to pass to do the thing, and forgetting how to run old projects. Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Marc - Confused about Snapshots (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/393/feedback/Marc%20-%20Confused%20about%20Snapshots) Dan’s gist: https://gist.github.com/dlangille/3140e60a816226ed75365ba8af185085 Pete - A Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/393/feedback/Pete%20-%20A%20Question) Rick - ZFS Idea (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/393/feedback/Rick%20-%20ZFS%20Idea) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** Special Guest: Dan Langille.
Thu, 11 Mar 2021 00:00:00 -0800
392: macOS inspired Desktop
FreeBSD 13 BETA Benchmarks, FreeBSD Jails Deep Dive by Klara Systems, FreeBSD Foundation looking for a Senior Arm Kernel Engineer & OSS Project Coordinator, macOS-Inspired BSD Desktop OS by helloSystem, A Trip into FreeBSD and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines FreeBSD 13 BETA Benchmarks - Performance Is Much Better (https://www.phoronix.com/scan.php?page=article&item=freebsd-13-beta1&num=6) FreeBSD Jails – Deep Dive into the Beginning of FreeBSD Containers (https://klarasystems.com/articles/freebsd-jails-the-beginning-of-freebsd-containers/) In recent years, containers and virtualization have become a buzzword in the Linux community, especially with the rise of Docker and Kubernetes. What many people probably don’t realize is that these ideas have been around for a very long time. Today, we will be looking at Jails and how they became part of FreeBSD. News Roundup FreeBSD Jobs The FreeBSD Foundation is looking for a Senior Arm Kernel Engineer (https://www.fossjobs.net/job/10369/senior-arm-kernel-engineer-at-the-freebsd-foundation/) The FreeBSD Foundation is also looking for an Open Source Project Coordinator. (https://www.fossjobs.net/job/10367/freebsd-open-source-project-coordinator-at-freebsd/) *** ### helloSystem Releases New ISOs For This macOS-Inspired BSD Desktop OS (https://www.phoronix.com/scan.php?page=news_item&px=helloSystem-New-12.1-Exp-ISOs) > The helloSystem motto is being a "desktop system for creators with focus on simplicity, elegance, and usability. Based on FreeBSD. Less, but better!" The desktop utilities are written with PyQt5. *** ### A Trip into FreeBSD (https://christine.website/blog/a-trip-into-freebsd-2021-02-13) > I normally deal with Linux machines. Linux is what I know and it's what I've been using since I was in college. A friend of mine has been coaxing me into trying out FreeBSD, and I decided to try it out and see what it's like. Here's some details about my experience and what I've learned. *** ###Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Beastie Bits Testing Linux Steam Proton on GhostBSD with BSD linuxulator - NO Audio (https://www.youtube.com/watch?v=H8ihW0m3bRQ) New Build of DragonFlyBSD 5.8 (http://lists.dragonflybsd.org/pipermail/users/2021-February/381550.html) Install OpenBSD 6.8 on PINE64 ROCK64 Media Board (https://github.com/krjdev/rock64_openbsd) FOSDEM BSD Track Videos are up (https://fosdem.org/2021/schedule/track/bsd/) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** Special Guest: Dan Langille.
Thu, 04 Mar 2021 00:00:00 -0800
391: i386 tear shedding
Follow-up about FreeBSD jail advantages, Install Prometheus, Node Exporter and Grafana, Calibrate your touch-screen on OpenBSD, OPNsense 21.1 Marvelous Meerkat Released, NomadBSD 1.4-RC1, Lets all shed a Tear for 386, find mostly doesn't need xargs today on modern Unixes, OpenBSD KDE Status Report, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Follow-up about FreeBSD jail advantages (https://rubenerd.com/follow-up-about-freebsd-jail-advantages/) I’ll admit I ran a lot of justifications together into a single paragraph because I wanted to get to configuring the jails themselves. They’re also, by and large, not specific to FreeBSD’s flavour of containerisation, though I still think it’s easily the most elegant implementation. Sometimes the simplest solution really is the best one. History of FreeBSD part 4: TCP/IP (https://klarasystems.com/articles/history-of-freebsd-part-4-bsd-and-tcp-ip/) How TCP/IP evolved and BSDs special contribution to the history of the Internet *** FreeBSD: Install Prometheus, Node Exporter and Grafana (https://blog.andreev.it/?p=5289) FreeBSD comes out of the box with three great tools for monitoring. If you need more info about how these tools work, please read the official documentation. I’ll explain the installation only and creating a simple dashboard. News Roundup Calibrate your touch-screen on OpenBSD (https://www.tumfatig.net/20210122/calibrate-your-touch-screen-on-openbsd/) I didn’t expected it but my refurbished T460s came with a touch-screen. It is recognized by default on OpenBSD and not well calibrated as-is. But that’s really simple to solve. Lets all shed a Tear for 386 (https://lists.freebsd.org/pipermail/freebsd-announce/2021-January/002006.html) FreeBSD is designating i386 as a Tier 2 architecture starting with FreeBSD 13.0. The Project will continue to provide release images, binary updates, and pre-built packages for the 13.x branch. However, i386-specific issues (including SAs) may not be addressed in 13.x. The i386 platform will remain Tier 1 on FreeBSD 11.x and 12.x. OPNsense 21.1 Marvelous Meerkat Released (https://opnsense.org/opnsense-21-1-marvelous-meerkat-released/) For more than 6 years, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing. NomadBSD 1.4-RC1 (https://nomadbsd.org/index.html#1.4-RC1) We are pleased to present the first release candidate of NomadBSD 1.4. find mostly doesn't need xargs today on modern Unixes (https://utcc.utoronto.ca/~cks/space/blog/unix/FindWithoutXargsToday) I've been using Unix for long enough that 'find | xargs' is a reflex. When I started and for a long time afterward, xargs was your only choice for efficiently executing a command over a bunch of find results. OpenBSD KDE Status Report (https://undeadly.org/cgi?action=article;sid=20210124113220) OpenBSD has managed to drop KDE3 and KDE4 in the 6.8 -> 6.9 release cycle. That makes me very happy because it was a big piece of work and long discussions. This of course brings questions: Kde Plasma 5 package missing. After half a year of work, I managed to successfully update the Qt5 stack to the last LTS version 5.15.2. On the whole, the most work was updating QtWebengine. What a monster! With my CPU power at home, I can build it 1-2 times a day which makes testing a little bit annoying and time intensive. Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Karl - Firefox webcam audio solution (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/391/feedback/Karl%20-%20Firefox%20webcam%20audio%20solution.md) Michal - openzfs (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/391/feedback/Michal%20-%20openzfs.md) Dave - bufferbloat (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/391/feedback/Dave%20-%20bufferbloat.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 25 Feb 2021 00:00:00 -0800
390: Commercial Unix Killer
Did Linux kill Commercial Unix, three node GlusterFS setup on FreeBSD, OpenBSD on the Lenovo ThinkPad X1 Nano (1st Gen), NetBSD on EdgeRouter Lite, TLS Mastery first draft done NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Did Linux Kill Commercial Unix? (https://www.howtogeek.com/440147/did-linux-kill-commercial-unix/) Sales of commercial Unix have fallen off a cliff. There has to be something behind this dramatic decline. Has Linux killed its ancestor by becoming a perfectly viable replacement, like an operating system version of Invasion of the Body Snatchers? Wireguard: Simple and Secure VPN in FreeBSD (https://klarasystems.com/articles/simple-and-secure-vpn-in-freebsd/) A great article by Tom Jones about setting up Wireguard on FreeBSD *** Setup a Three Node Replicated GlusterFS Cluster on FreeBSD (http://www.unibia.com/unibianet/freebsd/setup-three-node-replicated-glusterfs-cluster-freebsd) GlusterFS (GFS) is the open source equivalent to Microsoft's Distributed Filesystem (DFS). It's a service that replicates the contents of a filesystem in real time from one server to another. Clients connect to any server and changes made to a file will replicate automatically. It's similar to something like rsync or syncthing, but much more automatic and transparent. A FreeBSD port has been available since v3.4, and (as of this post) is currently at version 8.0 with 9.0 being released soon. News Roundup OpenBSD on the Lenovo ThinkPad X1 Nano (1st Gen) (https://jcs.org/2021/01/27/x1nano) Lenovo has finally made a smaller version of its X1 Carbon, something I’ve been looking forward to for years. NetBSD on the EdgeRouter Lite (https://www.cambus.net/netbsd-on-the-edgerouter-lite/) NetBSD-current now has pre-built octeon bootable images (which will appear in NetBSD 10.0) for the evbmips port, so I decided to finally give it a try. I've been happily running OpenBSD/octeon on my EdgeRouter Lite for a few years now, and have previously published some notes including more detail about the CPU. “TLS Mastery” first draft done! (https://mwl.io/archives/9938) Beastie Bits A Thread on a FreeBSD Desktop for PineBook Pro (https://forums.freebsd.org/threads/freebsd-desktop-for-pinebook-pro.78269/) FOSSASIA Conference - March 2021(Virtual) (https://eventyay.com/e/fa96ae2c) WireGuard for pfSense Software (https://www.netgate.com/blog/wireguard-for-pfsense-software.html) NetBSD logo to going Moon (https://mail-index.netbsd.org/netbsd-advocacy/2021/02/07/msg000849.html) *** ###Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. ### Producer's Note > Hey everybody, it’s JT here. After our AMA episode where I mentioned I was looking for older BSD Retail Copies, I was contacted by Andrew who hooked me up with a bunch of OpenBSD disks from the 4.x era. So shout out to him, and since that worked so well, I figured I'd give it another shot and ask that if anyone has any old Unixes that will run on an 8088, 8086, or 286 and you're willing to send me copies of the disks. I've recently dug out an old 286 system and I’d love to get a Unix OS on it. I know of Minix, Xenix and Microport, but I haven’t been able to find many versions of them. I've found Microport 1.3.3, and SCO Xenix... but that's about it. Let me know if you happen to have any other versions, or know where I can get them. Feedback/Questions Christian - ZFS replication and verification (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/390/feedback/Christian%20-%20ZFS%20replication%20and%20verification) Iain - progress (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/390/feedback/Iain%20-%20progress) Paul - APU2 device (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/390/feedback/Paul%20-%20APU2%20device) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 18 Feb 2021 00:00:00 -0800
389: Comfy FreeBSD Jails
A week with Plan 9, Exploring Swap on FreeBSD, how to create a FreeBSD pkg mirror using bastille and poudriere, How to set up FreeBSD 12 VNET jail with ZFS, Creating Comfy FreeBSD Jails Using Standard Tools, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines A Week With Plan 9 (https://thedorkweb.substack.com/p/a-week-with-plan-9) I spent the first week of 2021 learning an OS called Plan 9 from Bell Labs. This is a fringe Operating System, long abandoned by it’s original authors. It's also responsible for a great deal of inspiration elsewhere. If you’ve used the Go language, /proc, UTF-8 or Docker, you’ve used Plan 9-designed features. This issue dives into Operating System internals and some moderately hard computer science topics. If that sort of thing isn’t your bag you might want to skip ahead. Normal service will resume shortly. Exploring Swap on FreeBSD (https://klarasystems.com/articles/exploring-swap-on-freebsd/) On modern Unix-like systems such as FreeBSD, “swapping” refers to the activity of paging out the contents of memory to a disk and then paging it back in on demand. The page-out activity occurs in response to a lack of free memory in the system: the kernel tries to identify pages of memory that probably will not be accessed in the near future, and copies their contents to a disk for safekeeping until they are needed again. When an application attempts to access memory that has been swapped out, it blocks while the kernel fetches that saved memory from the swap disk, and then resumes execution as if nothing had happened. News Roundup How to create a FreeBSD pkg mirror using bastille and poudriere (https://hackacad.net/post/2021-01-13-build-a-freebsd-pkg-mirror-with-bastille-poudriere/) This a short how-to for creating a FreeBSD pkg mirror using BastilleBSD and Poudriere. How to set up FreeBSD 12 VNET jail with ZFS (https://www.cyberciti.biz/faq/configuring-freebsd-12-vnet-jail-using-bridgeepair-zfs/) How do I install, set up and configure a FreeBSD 12 jail with VNET on ZFS? How can I create FreeBSD 12 VNET jail with /etc/jail.conf to run OpenVPN, Apache, Wireguard and other Internet-facing services securely on my BSD box? FreeBSD jail is nothing but operating system-level virtualization that allows partitioning a FreeBSD based Unix server. Such systems have their root user and access rights. Jails can use network subsystem virtualization infrastructure or share an existing network. FreeBSD jails are a powerful way to increase security. Usually, you create jail per services such as an Nginx/Apache webserver with PHP/Perl/Python app, WireGuard/OpeNVPN server, MariaDB/PgSQL server, and more. This page shows how to configure a FreeBSD Jail with vnet and ZFZ on FreeBSD 12.x. Creating Comfy FreeBSD Jails Using Standard Tools (https://kettunen.io/post/standard-freebsd-jails/) Docker has stormed into software development in recent years. While the concepts behind it are powerful and useful, similar tools have been used in systems for decades. FreeBSD’s jails in one of those tools which build upon even older chroot(2) To put it shortly, with these tools, you can make a safe environment separated from the rest of the system. Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Chris - USB BSD variant (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/389/feedback/Chris%20-%20USB%20BSD%20variant) Jacob - host wifi through a jail (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/389/feedback/Jacob%20-%20host%20wifi%20through%20a%20jail) Jordan - new tool vs updating existing tool (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/389/feedback/Jordan%20-%20new%20too%20vs%20updating%20existing%20tool) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Wed, 10 Feb 2021 00:00:00 -0800
388: Must-have security tool
FreeBSD Q4 2020 Status report, a must-have security tool from OpenBSD, Bastille Port Redirection and Persistence, FreeBSD Wall Display Computer, etymology of command-line tools, GhostBSD 21.01.15 Release Notes, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines FreeBSD quarterly status report for Q4 2020 (https://www.freebsd.org/news/status/report-2020-10-2020-12/) Block spammers/abusive IPs with Pf-badhost in OpenBSD. A 'must have' security tool! (https://undeadly.org/cgi?action=article;sid=20210119113425) Pf-badhost is a very practical, robust, stable and lightweight security script for network servers. It's compatible with BSD based operating systems such as {Open,Free,Net,Dragonfly}BSD and MacOS. It prevents potentially-bad IP addresses that could possibly attack your servers (and waste your bandwidth and fill your logfiles), by blocking all those IPs contacting your server, and therefore it makes your server network/resources lighter and the logs of important services running on your server become simpler, more readable and efficient. News Roundup Bastille Port Redirection and Persistence (https://bastillebsd.org/blog/2021/01/13/bastille-port-redirection-and-persistence/) Bastille supports redirecting (rdr) ports from the host system into target containers. This port redirection is commonly used when running Internet services such as web servers, dns servers, email and many others. Any service you want to make public outside of your cluster will likely require port redirection (with some exceptions, see below). FreeBSD Wall Display Computer (https://blog.tyk.nu/blog/freebsd-wall-display-computer/) I've recently added a wall mounted 30" monitor for Grafana in my home. I can highly recommend doing the same, especially in a world where more work from home is becoming the norm. The etymology of command-line tools (https://i.redd.it/sni9gaxfj2d61.png) GhostBSD 21.01.15 Release Notes (https://ghostbsd.org/21.01.15_release_notes) I am happy to announce the availability of the new ISO 21.01.15. This new ISO comes with a clean-up of packages that include removing LibreOffice and Telegram from the default selection. We did this to bring the zfs RW live file systems to run without problem on 4GB of ram machine. We also removed the UFS full disk option from the installer. Users can still use custom partitions to setup UFS partition, but we discourage it. We also fixed the Next button's restriction in the custom partition related to some bug that people reported. We also fix the missing default locale setup and added the default setup for Linux Steam, not to forget this ISO includes kernel, userland and numerous application updates. Beastie Bits Interview with Brian Kernighan (https://corecursive.com/brian-kernighan-unix-bell-labs1/) *** ###Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 04 Feb 2021 00:00:00 -0800
387: OpenBSD Broadcast Studio
GNN's tips for surviving Cabin Fever and Coding from Home, Self-host a password manager on OpenBSD, Preliminary OpenBSD Support added to OBS, Dan's CURL tip of the Day, List of some Shell goodies for OpenBSD, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines GNN's tips for surviving Cabin Fever and Coding from Home (https://queue.acm.org/detail.cfm?id=3437846) Forgive me if this seems off topic, but I was wondering if you had any advice for the majority of us who are now KFH (koding from home). I don't know how KV works day to day, but it seems pretty clear that the status quo has changed at most workplaces in the last several months, and it's hard to know if there are things we could be doing to stay productive while we're all at home, ordering delivery, and microwaving our mail. Does KV have any good guidance? Self-host a password manager on OpenBSD (https://www.tumfatig.net/20210105/self-host-a-password-manager-on-openbsd/) I’ve been using Rubywarden to store and access my passwords from OpenBSD workstations and iOS toys. But recent redondant failures from the iOS App and rubywarden not being maintained anymore led to the need for a new solution. I was investing on pass+pgp+git but it was quite complex. News Roundup Preliminary OpenBSD Support added to OBS (https://undeadly.org/cgi?action=article;sid=20210113072623) Dan's CURL tip of the Day (https://mobile.twitter.com/DLangille/status/1323963716153626626) List of some Shell goodies for OpenBSD (https://www.vincentdelft.be/post/post_20210102) I'm sharing here some practices I'm following and some small tips/tools which facilitate my usage of OpenBSD in my day to day. Some are really specific to my usage, others could be re-used. Beastie Bits • [Traditional text mode games from BSD](https://github.com/msharov/bsd-games) • [FreeBSD Easter Eggs](https://twitter.com/freebsdfrau/status/972893680473317377) • [A prehistory and history of Unix Slide Deck](https://docs.google.com/presentation/d/1BxnFiP_Hv3HJbbYRfSxpTym7GzqxJPQlTE6Ur5h1Al8/edit#slide=id.g951f86c343_0_95) • [How to use Android USB Tethering to get Internet on FreeBSD](https://www.youtube.com/watch?v=cAEmtrEZlV8) • [VPN'Othon #2 for CharmBUG](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/387/charmbug_event.md) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions • [Kev - Ramdisk](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/387/feedback/kev%20-%20ramdisk.md) • [John - new to freebsd](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/387/feedback/John%20-%20new%20to%20freebsd) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 28 Jan 2021 00:00:00 -0800
386: Aye, 386!
Routing and Firewalling VLANS with FreeBSD, FreeBSD 12 VNET jail with ZFS howto, pkgsrc-2020Q4 released, FreeBSD on Raspberry Pi 4 With 4GB of RAM, HardenedBSD December 2020 Status Report, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Routing and Firewalling VLANS with FreeBSD (https://klarasystems.com/articles/routing-and-firewalling-vlans-with-freebsd/) In this article we are going to look at and integrate two network isolation technologies, VLANs and VNET. VLANs are common place, and if you have done some network management or design then you are likely to have interacted with them. The second are FreeBSDs VNET virtual network stacks, a powerful network stack isolation technology that gives FreeBSD jails super powers. Ethernet VLAN (standardised by IEEE 802.1Q) are an extension to Ethernet and provide an essential method for scaling network deployments. They are used in all environments to enable reuse of common infrastructure by isolating portions of networks from each other. VLANs allow the reuse of common cables, switches and routers to carry completely different networks. It is common to have data that must be separated from different networks carried on common cables until their VLAN tags are finally stripped at a gateway switch or router. How to set up FreeBSD 12 VNET jail with ZFS (https://www.cyberciti.biz/faq/configuring-freebsd-12-vnet-jail-using-bridgeepair-zfs/) How do I install, set up and configure a FreeBSD 12 jail with VNET on ZFS? How can I create FreeBSD 12 VNET jail with /etc/jail.conf to run OpenVPN, Apache, Wireguard and other Internet-facing services securely on my BSD box? FreeBSD jail is nothing but operating system-level virtualization that allows partitioning a FreeBSD based Unix server. Such systems have their root user and access rights. Jails can use network subsystem virtualization infrastructure or share an existing network. FreeBSD jails are a powerful way to increase security. Usually, you create jail per services such as an Nginx/Apache webserver with PHP/Perl/Python app, WireGuard/OpeNVPN server, MariaDB/PgSQL server, and more. This page shows how to configure a FreeBSD Jail with vnet and ZFS on FreeBSD 12.x. News Roundup pkgsrc-2020Q4 released (https://mail-index.netbsd.org/netbsd-announce/2021/01/08/msg000322.html) The pkgsrc developers are proud to announce the 69th quarterly release of pkgsrc, the cross-platform packaging system. pkgsrc is available with more than 24,000 packages, running on 23 separate platforms; more information on pkgsrc itself is available at https://www.pkgsrc.org/ FreeBSD ON A Raspberry PI 4 With 4GB of RAM (https://lambdaland.org/posts/2020-12-23_freebsd_rpi4/) This is the story of how I managed to get FreeBSD running on a Raspberry Pi 4 with 4GB of RAM, though I think the setup story is pretty similar for those with 2GB and 8GB.1 HardenedBSD December 2020 Status Report (https://hardenedbsd.org/article/shawn-webb/2020-12-31/hardenedbsd-december-2020-status-report) Happy New Year! On this the last day of 2020, I submit December's status report. Beastie Bits Christmas Cards The Unix Way - with pic and troff (https://www.youtube.com/watch?v=xMijdTWSUEE&feature=youtu.be) Fast RPI3 upgrade from source (cross compile) (https://forums.freebsd.org/threads/fast-upgrade-raspberry-pi3-from-source.78169/) *** ###Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Robert - zfs question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/386/feedback/robert%20-%20zfs%20question.md) Neb - AMA episode.md (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/386/feedback/neb%20-%20AMA%20episode.md) Joe - puppet (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/386/feedback/joe%20-%20puppet.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 21 Jan 2021 00:00:00 -0800
385: Wireguard VPN mesh
Description: History of FreeBSD: Early Days of FreeBSD, mesh VPN using OpenBSD and WireGuard, FreeBSD Foundation Sponsors LLDB Improvements, Host your Cryptpad web office suite with OpenBSD, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines History of FreeBSD - Part 3: Early Days of FreeBSD (https://klarasystems.com/articles/history-of-freebsd-part-3-early-days-of-freebsd/?utm_source=bsdnow) In this third part of our series on the history of FreeBSD, we start tracing the early days of FreeBSD and the events that would eventually shape the project and the future of open source software. A mesh VPN using OpenBSD and WireGuard (https://www.tumfatig.net/20201202/a-mesh-vpn-using-openbsd-and-wireguard/?utm_source=bsdnow) WireGuard is a new coming to OpenBSD 6.8 and it looks like a simple and efficient way to connect computers. I own a few VPS (hello Vultr, hello OpenBSD.amsterdam) that tend to be connected through filtered public services and/or SSH tunnels. And that’s neither efficient nor easy to manage. Here comes the wg(4) era where all those peers will communicate with a bit more privacy and ease of management. News Roundup Foundation Sponsors FreeBSD LLDB Improvements (https://freebsdfoundation.org/blog/guest-blog-foundation-sponsors-freebsd-lldb-improvements/?utm_source=bsdnow) With FreeBSD Foundation grant, Moritz Systems improved LLDB support for FreeBSD The LLDB project builds on libraries provided by LLVM and Clang to provide a great modern debugger. It uses the Clang ASTs and the expression parser, LLVM JIT, LLVM disassembler, etc so that it provides an experience that “just works”. It is also blazing fast and more permissively licensed than GDB, the GNU Debugger. LLDB is the default debugger in Xcode on macOS and supports debugging C, Objective-C, and C++ on the desktop and iOS devices and the simulator. Host your Cryptpad web office suite with OpenBSD (https://dataswamp.org/~solene/2020-12-14-cryptpad-openbsd.html) In this article I will explain how to deploy your own Cryptpad instance with OpenBSD. Cryptpad is a web office suite featuring easy real time collaboration on documents. Cryptpad is written in JavaScript and the daemon acts as a web server. Beastie Bits OPNsense 20.7.7 Released (https://opnsense.org/opnsense-20-7-7-released/?utm_source=bsdnow) Introducing OpenZFS 2.0 Webinar - Jan 20th @ noon Eastern / 17:00 UTC. (https://klarasystems.com/learning/webinars/webinar-introducing-openzfs-2-0/?utm_source=bsdnow) BSD In Die Hard (https://www.reddit.com/r/BSD/comments/kk3c6y/merry_xmas/) Managing jails with Ansible: a showcase for building a container infrastructure on FreeBSD (https://papers.freebsd.org/2019/bsdcan/dengg-managing_jails_with_ansible/) BSD Hardware (https://bsd-hardware.info) New WINE chapter in FreeBSD handbook (https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/wine.html) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. *** Feedback/Questions scott- zfs question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/385/feedback/scott-%20zfs%20question) Bruce - copy paste on esxi (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/385/feedback/Bruce%20-%20copy%20paste%20on%20esxi) Julian - an apology for Allan (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/385/feedback/Julian%20-%20an%20apology%20for%20Allan) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 14 Jan 2021 00:00:00 -0800
384: In memoriam
Allen K. Briggs Memorial Scholarship, Toward an automated tracking of OpenBSD ports contributions, Trying OpenZFS 2 on FreeBSD 12.2-RELEASE, OpenBSD on TECLAST F7 Plus, Multi-volume support in HAMMER2, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Allen K. Briggs Memorial Scholarship (http://blog.netbsd.org/tnf/entry/allen_k_briggs_memorial_scholarship) Allen Briggs was one of the earliest members of the NetBSD community, pursuing his interest in macBSD, and moving to become a NetBSD developer when the two projects merged. Allen was known for his quiet and relaxed manner, and always brought a keen wisdom with him; allied with his acute technical expertise, he was one of the most valued members of the NetBSD community. The Allen K. Briggs Memorial Scholarship is an endowment to provide scholarships in perpetuity for summer programs at the North Carolina School of Science & Math, which Allen considered to be a place that fundamentally shaped him as a person. We would love to invite Allen's friends and colleagues from the BSD community to donate to this cause so that we can provide more scholarships to students with financial need each year. We are approximately halfway to our goal of $50K with aspirations to exceed that target and fund additional scholarships. Toward an automated tracking of OpenBSD ports contributions (https://dataswamp.org/~solene/2020-11-15-openbsd-ports-ci.html) A first step for the CI service would be to create a database of diffs sent to ports. This would allow people to track what has been sent and not yet committed and what the state of the contribution is (build/don’t build, apply/don’t apply). News Roundup Trying OpenZFS 2 on FreeBSD 12.2-RELEASE (https://rubenerd.com/trying-openzfs-on-freebsd-12-release/?utm_source=bsdnow) OpenZFS 2 is a huge achievement, and makes me bullish about the long term prospects for the world’s most trustworthy and nicest to use storage system. You can even use try it today on FreeBSD 12.2-RELEASE, though I recommend tracking -CURRENT for these sorts of features. OpenBSD on TECLAST F7 Plus (https://www.tumfatig.net/20201215/openbsd-on-teclast-f7-plus/?utm_source=bsdnow) I got myself a TECLAST F7 Plus laptop. It comes preinstalled with Windows 10 but I planned to use it as my daily driver. So I installed OpenBSD 6.8 on it. Multi-volume support in HAMMER2 (https://www.dragonflydigest.com/2020/12/28/25287.html) commit (http://lists.dragonflybsd.org/pipermail/commits/2020-December/770072.html) > This commit adds initial multi-volumes support for HAMMER2. Maximum supported volumes is 64. The feature and implementation is similar to multi-volumes support in HAMMER1. *** Beastie Bits FreeBSD Last SVN Commit (https://svnweb.freebsd.org/base/head/README?view=markup&pathrev=368820) FreeBSD First git Commit (https://cgit.freebsd.org/src/commit/?id=5ef5f51d2bef80b0ede9b10ad5b0e9440b60518c) Introducing OpenZFS 2.0 Webinar - Jan 20th @ noon Eastern / 17:00 UTC. (https://klarasystems.com/learning/webinars/webinar-introducing-openzfs-2-0/?utm_source=bsdnow) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. *** Feedback/Questions jay - feedback for ian (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/384/feedback/jay%20-%20feedback%20for%20ian) Iebluefire - concerns about freebsd (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/384/feedback/Iebluefire%20-%20concerns%20about%20freebsd) mike - zfs cluster aware (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/384/feedback/mike%20-%20zfs%20cluster%20aware) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 07 Jan 2021 00:45:00 -0800
383: Scale the tail
FreeBSD Remote Process Plugin Final Milestone achieved, Tailscale for OpenBSD, macOS to FreeBSD migration, monitoring of our OpenBSD machines, OPNsense 20.7.6 released, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines FreeBSD Remote Process Plugin: Final Milestone Achieved (https://www.moritz.systems/blog/freebsd-remote-plugin-final-milestone-achieved/) Moritz Systems have been contracted by the FreeBSD Foundation to modernize the LLDB debugger’s support for FreeBSD. We are working on a new plugin utilizing the more modern client-server layout that is already used by Darwin, Linux, NetBSD and (unofficially) OpenBSD. The new plugin is going to gradually replace the legacy one. Tailscale on OpenBSD (https://rakhesh.com/linux-bsd/tailscale-on-openbsd/) I spent some time setting this up today evening and thought I’d post the steps here. Nothing fancy, just putting together various pieces actually. I assume you know what Tailscale is; if not check out their website. Basically it is a mesh network built on top of Wireguard. Using it you can have all your devices both within your LAN(s) and outside be on one overlay network as if they are all on the same LAN and can talk to each other. It’s my new favourite thing! News Roundup macOS to FreeBSD migration a.k.a why I left macOS (https://antranigv.am/weblog_en/posts/macos_to_freebsd/) This is not a technical documentation for how I migrated from macOS to FreeBSD. This is a high-level for why I migrated from macOS to FreeBSD. Not so long ago, I was using macOS as my daily driver. The main reason why I got a macbook was the underlying BSD Unix and the nice graphics it provides. Also, I have an iPhone. But they were also the same reasons for why I left macOS. Our monitoring of our OpenBSD machines, such as it is (as of November 2020 (https://utcc.utoronto.ca/~cks/space/blog/sysadmin/OurOpenBSDMonitoring) We have a number of OpenBSD firewalls in service (along with some other OpenBSD servers for things like VPN endpoints), and I was recently asked how we monitor PF and overall network traffic on them. I had to disappoint the person who asked with my answer, because right now we mostly don't (although this is starting to change). OPNsense 20.7.6 released (https://opnsense.org/opnsense-20-7-6-released/) This update brings the usual mix of reliability fixes, plugin and third party software updates: FreeBSD, HardenedBSD, PHP, OpenSSH, StrongSwan, Suricata and Syslog-ng amongst others. Please note that Let's Encrypt users need to reissue their certificates manually after upgrading to this version to fix the embedded certificate chain issue with the current signing CA switch going on. NYC Bug Jan 2021 with Michael W. Lucas (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/383/nycbug) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions cy - .so files (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/383/feedback/cy%20-%20.so%20files) ben - mixer volume (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/383/feedback/ben%20-%20mixer%20volume) probono - live cds (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/383/feedback/probono%20-%20live%20cds) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 31 Dec 2020 00:00:00 -0800
382: BSDNow Q&A 2020
We asked for it, you answered our call. This episode features you interviewing us with questions that you sent in. JT, Allan, and Benedict answer everything that you ever wanted to know in this week’s special episode of BSDNow. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Interview - Allan Jude - Allan.jude@gmail.com (Allan.jude@gmail.com) / @allanjude (https://twitter.com/allanjude) Interview - Benedict Reuschling - bcr@freebsd.org (mailto:bcr@freebsd.org) / @bsdbcr (https://twitter.com/bsdbcr) Interview - JT Pennington - jt@obs-sec.com (mailto:jt@obs-sec.com) / @q5sys (https://twitter.com/q5sys) AMA questions Benedict: You work at a university right? Were you already into tech before you started working there? What do you do there? Yes, I do work at the University of Applied Sciences, Darmstadt, Germany. I’m a lab engineer there (without a lab, but with a big data cluster). I teach in the winter semester an undergraduate, elective course called “Unix for Developers”. Yes, I was already in tech by that time. Did some previous work at companies before (selling hardware at the call-in hotline and later in the store) and during my CS studies. Allan: What’s the next big FreeBSD Project you plan on doing? JT: How did you get involved in BSD? Weren't you a Linux guy? All: Is there any way you can create an entire episode of BSDnow on hardware that runs OpenBSD and FreeBSD? We see you audacity, etc on a mac. Benedict: Not sure about OpenBSD (don’t use it), but FreeBSD should be doable for my part. If we switch from Skype to a different video chat tool, the rest is already there. Production side may be more difficult, but not impossible. All: if you could finish up one project right now... what would it be? Benedict: Updated ZFS chapter in the FreeBSD handbook. All: How did all of you guys meet? All: My question is, do you guys use FreeBSD as your main desktop OS? If not, what do you use? Benedict: No, but Mac OS is close enough. Doing a lot of SSHing into FreeBSD from there. All: Can you all give us the best shot of outside of their windows? JT’s answer: https://photos.smugmug.com/photos/i-2LSbspL/0/69437dbb/5K/i-2LSbspL-5K.jpg Allan: https://photos.app.goo.gl/UnKXnKMt6cn8FDhNA Benedict: No, it’s dark outside anyway. ;-) All: How old were you when you got your first computer and what was that computer? Allan: 12 or 13, a 486DX2/66hz with an insane 32mb of RAM, 400 and 500 MB SCSI HDDs, 14400 baud model, and a 1.7x CD rom drive Benedict: Around 13 or so. 386DX2, 4 MB RAM, IDE disk drive (no idea how big, but it wasn’t much), 3.5” floppy, DOS, and a lot of games. JT: Technically the first was a Atari 1200XL with a 6502 CPU running at 1.79 MHz 64KB RAM. It had it's own OS and you could load programs off of either cartridges, floppy disks, or cassette tapes. First PC Clone was a Packard Bell with a 386 and 1mb ram which later was upgraded to 4mb and a Dual speed CD-ROM. My dad got me a Compaq 286 laptop... this one (show)... a year or so later because he got tired of fighting me for the computer. All: Can we have a peek at your bookcase and what books are there? Allan: No picture handy, but my shelf is pretty small, mostly a collection of autographed FreeBSD books. I have D&I with all 3 autographs (took some travel to acquire), and a copy of my first book (FreeBSD Mastery: ZFS) autographed by Jeff Bonwick and Matt Ahrens, the creators of ZFS, plus a bunch of other big names in ZFS like George Wilson. JT’s answer: So... my library is packed away... but here’s about half of it... the rest is still in storage. https://photos.smugmug.com/photos/i-SBG2KDv/0/0b9856b8/4K/i-SBG2KDv-4K.jpg Software Collection: https://photos.smugmug.com/photos/i-HfTVPN9/0/ad610dd4/O/i-HfTVPN9.jpg Benedict: A mix of FreeBSD books (by MWL), the graveyard book, 4 hour work week, the once and future king (took me a long time to finish that one), Total Immersion swimming (still learning to swim) and some books in german language, fiction and tech. Groff lives in there while the pandemic lasts. All: What desktop/Window Manager/shell do each of you primarily use? Benedict: Mainly Mac OS, when on FreeBSD it’s i3. Zsh with zsh-autosuggestions currently. JT: Lumina/zsh Allan: Lumina and tcsh, want to learn zsh but never gotten time to change All: What spoken languages do you speak? Benedict: German and English (obviously), learning a bit of Spanish via Duolingo at the moment JT: English, Bad English, and some French. All: Do you have Non-Computer hobbies if so what are those? Benedict: Tai Chi Chuan (Yang Style) JT: I'd say photography, but that's a job for me. I have a lot of varied interests, Krav Maga, working on my VW Corrado, working on the old Victorian house I bought, and camping/backpacking. Ive done the northern half of the AT (Appalachian Trail, I want to finish it up and then do the PCT and CDT. (Pacific Crest Trail and Continental Divide Trail). All: When COVID passes, when are either of you are coming to BSD pizza night in Portland, OR, USA so I can buy you a beer/wine/whisky or pizza/coffee/tea (or six) Rapid Fire: All: What was the first car you ever owned? All: Do you own a vehicle and if so what is the make/model? All: Favorite Star franchise? Star Wars, Star Trek, Stargate, Battlestar, etc. JT: Will you ever host any more BSDNow episodes? All: Favorite superhero? Marvel and/or DC. All: Favorite game(s) of all time? All: Pants or no pants on virtual meetings/presentations? All: Do you or have you used alternative operating systems that are not "main stream or is considered retro" if so what are those? All: Who has more animals at home? Allan: Does Allan have any batteries for his tetris cubes? Can we see that thing light up? Allan and Benedict: Are you guys going to go on JT's new show? If you’re wondering what show this is, here are the two shows Im a host of: https://www.opensourcevoices.org & https://www.theopiniondominion.org Allan and Benedict: Have Allan or Benedict lost anything on the way to and from a conference? Benedict: Is Benedict going to do his NOEL blocks again? Benedict: Does Benedict make his bed every Wednesday morning? It always looks great! Not just Wednesdays, but pretty much every day. Here, watch this: https://www.youtube.com/watch?v=GKZRFDCbGTA Nuff said. ;-) JT: Are you batman because the episodes are always awesome sir so thank you JT’s answer: Can you ever admit to being batman? If I were batman wouldn't I have to deny it? All: What's your Daily Driver Hardware? All: Who has more servers or VMs at home? Benedict: Allan, easily JT: Allan definitely beats me with VMs, but I think I might give him a run on servers. 4x 4u HP DL580s, one HP DL980, three HP C3000 8 bay bladecenters, three HP C7000 16 bay Bladecenters, 2x Sun 280R, bunch of Dell and IBM 1Us… but all my stuff is old. Allan has all the new and shiny stuff. The Pile in the Kitchen: https://photos.smugmug.com/photos/i-HBScrpk/0/4b058cc5/X2/i-HBScrpk-X2.jpg The other pile: https://photos.smugmug.com/photos/i-wNxFszV/0/e7a4b2d6/X2/i-wNxFszV-X2.jpg All: What book(s) are you currently reading? Benedict: Antifragile by Nassim Taleb JT: Douglas Hofstader - Gödel, Escher, Bach: An Eternal Golden Braid. Douglas Rushkoff - program or be programmed. Also a 4 part book series on the American civil war written in the 1880s, by people in the civil war. All: Favorite mechanical keyboard switch? Cherry MX, Kalih, Gateron, etc. Benedict: Cherry MX brown currently Allan: Cherry MX Blue (Coolermaster Master Keys Pro-L) JT: I prefer scissor switches, so I use a Logitech K740. Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 24 Dec 2020 03:00:00 -0800
381: Shell origins
The Origin of the Shell, Return to Plan 9, ArisbluBSD: Why a new BSD?, OPNsense 20.7.5 released, Midnight BSD 2.0 Release Status, HardenedBSD November 2020 Status Report, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines The Origin of the Shell (https://multicians.org/shell.html) CTSS was developed during 1963 and 64. I was at MIT on the computer center staff at that time. After having written dozens of commands for CTSS, I reached the stage where I felt that commands should be usable as building blocks for writing more commands, just like subroutine libraries. Hence, I wrote "RUNCOM", a sort of shell driving the execution of command scripts, with argument substitution. The tool became instantly most popular, as it became possible to go home in the evening while leaving behind long runcoms executing overnight. It was quite neat for boring and repetitive tasks such as renaming, moving, updating, compiling, etc. whole directories of files for system and application maintenance and monitoring. Return to Plan 9 (https://boxbase.org/entries/2020/nov/1/return-to-plan9/) Plan 9 from Bell Labs has held the same charm after my last visit that took a few days. This time I'll keep this operating system in an emulator where I can explore into it when I am distracted. News Roundup Why a new BSD? (https://blog.fivnex.co/2020/11/arisblubsd-why-new-bsd.html) This article is to explain some decisions and plans made by the ArisbluBSD team, why we are making our own thing, and what the plan is for the OS. We mainly want to talk about five things: desktop, package management, software availability, custom software, and the future of the OS. We mostly want to explain what the goal of the OS is, and how we plan to expand in the near future. Without further ado, let's explain ArisbluBSD's plan. OPNsense 20.7.5 released (https://opnsense.org/opnsense-20-7-5-released/) We return briefly for a small patch set and plan to pin the 20.1 upgrade path to this particular version to avoid unnecessary stepping stones. We wish you all a healthy Friday. And of course: patch responsibly! Midnight BSD 2.0 Release Status (https://www.justjournal.com/users/mbsd/entry/33841) We identified some issues with the 2.0 ISOs slated for release with the ZFS bootloader not working. Until this issue is resolved, we are unable to build release ISOs. We've left the old ones up as they work fine for anyone using UFS. HardenedBSD November 2020 Status Report (https://hardenedbsd.org/article/shawn-webb/2020-11-25/hardenedbsd-november-2020-status-report) We're getting close to the end of November. My wife and I have plans this weekend, so I thought I'd take the time to write November's status report today. Beastie Bits • [rga: ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz, etc.](https://phiresky.github.io/blog/2019/rga--ripgrep-for-zip-targz-docx-odt-epub-jpg/) • [exa - A modern replacement for ls](https://the.exa.website/) • [The myriad meanings of pwd in Unix systems](https://qmacro.org/2020/11/08/the-meaning-of-pwd-in-unix-systems/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Karl - Camera Help (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/381/feedback/Karl%20-%20camera%20help.md) Alejandro - domain registrar (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/381/feedback/alejandro%20-%20domain%20registrar.md) Johnny - thoughts on 372 (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/381/feedback/Johnny%20-%20thoughts%20on%20372) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 17 Dec 2020 03:00:00 -0800
380: Early ZFS-mas
We read FreeBSD’s 3rd quarter status report, OpenZFS 2.0, adding check-hash checks in UFS filesystem, OpenSSL 3.0 /dev/crypto issues on FreeBSD, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines 3rd Quarter FreeBSD Report (https://www.freebsd.org/news/status/report-2020-07-2020-09.html) The call for submissions for the 4th Quarter is out (https://lists.freebsd.org/pipermail/freebsd-quarterly-calls/2020/000007.html) OpenZFS 2.0 (https://arstechnica.com/gadgets/2020/12/openzfs-2-0-release-unifies-linux-bsd-and-adds-tons-of-new-features/) This Monday, ZFS on Linux lead developer Brian Behlendorf published the OpenZFS 2.0.0 release to GitHub. Along with quite a lot of new features, the announcement brings an end to the former distinction between "ZFS on Linux" and ZFS elsewhere (for example, on FreeBSD). This move has been a long time coming—the FreeBSD community laid out its side of the roadmap two years ago—but this is the release that makes it official. News Roundup Revision 367034 (https://svnweb.freebsd.org/changeset/base/367034) Various new check-hash checks have been added to the UFS filesystem over various major releases. Superblock check hashes were added for the 12 release and cylinder-group and inode check hashes will appear in the 13 release. OpenSSL 3.0 /dev/crypto issues on FreeBSD (https://rubenerd.com/openssl-3-written-to-break-on-freebsd/) So, just learned that the OpenSSL devs decided to break /dev/crypto on FreeBSD. OS108-9.1 XFCE amd64 released (https://forums.os108.org/d/32-os108-91-xfce-amd64-released) OS108 is a fast, open and Secure Desktop Operating System built on top of NetBSD. > Installing OS108 to your hard drive is done by using the sysinst utility, the process is basically the same as installing NetBSD itself. Please refer to the NetBSD guide for installation details, http://www.netbsd.org/docs/guide/en/part-install.html Installation Video (https://youtu.be/cgAeY21gXR4) *** Beastie Bits OpenBGPD 6.8p1 portable: released Nov 5th, 2020 (http://www.openbgpd.org/ftp.html) IRC Awk Bot (http://kflu.github.io/2020/08/15/2020-08-15-awk-irc-bot/) Docker on FreeBSD using bhyve and sshfs (https://www.youtube.com/watch?v=ZVkJZJEdZNY) The UNIX Command Language (1976) (https://github.com/susam/tucl) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions santi - openrc (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/380/feedback/santi%20-%20openrc.md) trond - python2 and mailman (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/380/feedback/trond%20-%20python2%20and%20mailmane%20and%20sshfs) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 10 Dec 2020 03:00:00 -0800
379: bhyve my guest
Adventures in Freebernetes, tracing kernel functions, The better way of building FreeBSD networks, New beginnings: CDBUG virtual meetings, LibreSSL update in DragonFly, Signal-cli with scli on FreeBSD, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Adventures in Freebernetes: bhyve My Guest (https://productionwithscissors.run/2020/10/29/adventures-in-freebernetes-bhyve-my-guest/) Part 2 of experiments in FreeBSD and Kubernetes: Creating your first guest Tracing Kernel Functions: FBT stack() and arg (https://zinascii.com/2020/fbt-args-and-stack.html?s=03) In my previous post I described how FBT intercepts function calls and vectors them into the DTrace framework. That laid the foundation for what I want to discuss in this post: the implementation of the stack() action and built-in arg variables. These features rely on the precise layout of the stack, the details of which I touched on previously. In this post I hope to illuminate those details a bit more with the help of some visuals, and then guide you through the implementation of these two DTrace features as they relate to the FBT provider. News Roundup Dummynet: The Better Way of Building FreeBSD Networks (https://klarasystems.com/articles/dummynet-the-better-way-of-building-freebsd-networks/) Dummynet is the FreeBSD traffic shaper, packet scheduler, and network emulator. Dummynet allows you to emulate a whole set of network environments in a straight-forward way. It has the ability to model delay, packet loss, and can act as a traffic shaper and policer. Dummynet is roughly equivalent to netem in Linux, but we have found that dummynet is easier to integrate and provides much more consistent results. New beginnings: CDBUG virtual meetings (http://lists.nycbug.org/pipermail/cdbug-talk/2020-October/000901.html) I had overwhelmingly positive responses from the broader *BSD community about restarting CDBUG meetings as virtual, at least for now. Hopefully this works well and even when we're back to in-person meetings we can still find a way to bring in virtual attendees. LibreSSL update in DragonFly (https://www.dragonflydigest.com/2020/11/10/25143.html) DragonFly has a new version of libressl, noting cause it has a newer TLS1.3 implementation – something that may be necessary for you. Signal-cli with scli on FreeBSD (https://antranigv.am/weblog_en/posts/freebsd-signal-cli-scli/) So couple of days ago I migrated from macOS on Macbook Pro to FreeBSD on ThinkPad T480s. Beastie Bits Firefox is not paxctl safe for NetBSD (https://anonhg.netbsd.org/pkgsrc/rev/9386adbd052e) FreeBSD 12.2-RELEASE on Microsoft Azure Marketplace (https://azuremarketplace.microsoft.com/en-us/marketplace/apps/thefreebsdfoundation.freebsd-12_2?tab=Overview) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions carlos - BSD Now around the world (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/379/feedback/carlos%20-%20BSD%20Now%20around%20the%20world.md) paulo - freebsd on a Bananapi (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/379/feedback/paulo%20-%20freebsd%20on%20a%20Bananapi.md) paulo - followup (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/379/feedback/paulo%20-%20followup.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 03 Dec 2020 03:45:00 -0800
378: Networknomicon
Interview with Michael W. Lucas: SNMP and TLS book, cashflow for creators, book sale and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Interview with Michael W. Lucas SNMP Book (https://mwl.io/nonfiction/networking#snmp) The Networknomicon (https://mwl.io/nonfiction/networking#networknomicon) Sponsor the TLS Book (https://www.tiltedwindmillpress.com/product-category/sponsor/) Cashflow for creators (https://mwl.io/nonfiction/biz-craft) Book sale (https://mwl.io/blog/9313) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** Special Guest: Michael W Lucas.
Thu, 26 Nov 2020 03:00:00 -0800
377: Firewall ban-sharing
History of FreeBD: BSDi and USL Lawsuits, Building a Website on Google Compute Engine, Firewall ban-sharing across machines, OpenVPN as default gateway on OpenBSD, Sorting out what the Single Unix Specification is, Switching from Apple to a Thinkpad for development, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines History of FreeBSD : Part 2 : BSDi and USL Lawsuits (https://klarasystems.com/articles/history-of-freebsd-part-2-bsdi-and-usl-lawsuits/) In this second part of our series on the history of FreeBSD, we continue to trace the pre-history of FreeBSD and the events that would eventually shape the project and the future of open source software. Building a Web Site on Google Compute Engine (https://cromwell-intl.com/open-source/google-freebsd-tls/) Here's how I deployed a web site to the Google Cloud Platform. I used FreeBSD for good performance, stability, and minimal complexity. I set up HTTPS with free Let's Encrypt TLS certificates for both RSA and ECC. Then I adjusted the Apache configuration for a good score from the authoritative Qualys server analysis. News Roundup Firewall ban-sharing across machines (https://chown.me/blog/acacia) As described in My infrastructure as of 2019, my machines are located in three different sites and are loosely coupled. Nonetheless, I wanted to set things up so that if an IP address is acting maliciously toward one machine, all my machines block that IP at once so the meanie won't get to try one machine after another. OpenVPN as default gateway on OpenBSD (https://dataswamp.org/~solene/2020-10-27-openbsd-openvpn.html) If you plan to use an OpenVPN tunnel to reach your default gateway, which would make the tun interface in the egress group, and use tun0 in your pf.conf which is loaded before OpenVPN starts? Here are the few tips I use to solve the problems. Sorting out what the Single Unix Specification is and covers (https://utcc.utoronto.ca/~cks/space/blog/unix/SingleUnixSpecificationWhat) Sorting out what the Single Unix Specification is and covers October 8, 2020 I've linked to the Single Unix Specification any number of times, for various versions of it (when I first linked to it, it was at issue 6, in 2006; it's now up to a 2018 edition). But I've never been quite clear what it covered and didn't cover, and how it related to POSIX and similar things. After yesterday's entry got me looking at the SuS site again, I decided to try to sort this out once and for all. Bye-bye, Apple (http://blog.cretaria.com/posts/bye-bye-apple.html) The days of Apple products are behind me. I had been developing on a Macbook for over twelve years, but now, I’ve switched to an ever trending setup: OpenBSD on a Thinkpad. The new platform is a winner. Everything is clean, quick, and configurable. When I ps uaxww, I’m not hogging ‘gigs’ of RAM just to have things up and running. There’s no black magic that derails me at every turn. In short, my sanity has been long restored. Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Chris - small projects (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/377/feedback/Chris%20-%20small%20projects.md) Jens - ZFS Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/377/feedback/Jens%20-%20ZFS%20Question.md) One pool to rule them all (https://ftfl.ca/blog/2016-09-17-zfs-fde-one-pool-conversion.html) Shroyer - Dotnet on FreeBSD for Jellyfin (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/377/feedback/Shroyer%20-%20Dotnet%20on%20FreeBSD%20for%20Jellyfin.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 19 Nov 2020 03:00:00 -0800
376: Build stable packages
FreeBSD 12.2 is available, ZFS Webinar, Enhancing Syzkaller support for NetBSD, how the OpenBSD -stable packages are built, OPNsense 20.7.4 released, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines FreeBSD 12.2 Release (https://www.freebsd.org/releases/12.2R/relnotes.html) The release notes for FreeBSD 12.2-RELEASE contain a summary of the changes made to the FreeBSD base system on the 12-STABLE development line. This document lists applicable security advisories that were issued since the last release, as well as significant changes to the FreeBSD kernel and userland. Some brief remarks on upgrading are also presented. ZFS Webinar: November 18th (https://klarasystems.com/learning/best-practices-for-optimizing-zfs1/) Join us on November 18th for a live discussion with Allan Jude (VP of Engineering at Klara Inc) in this webinar centred on “best practices of ZFS” Building Your Storage Array – Everything from picking the best hardware to RAID-Z and using mirrors. Keeping up with Data Growth – Expanding and growing your pool, and of course, shrinking with device evacuation. Datasets and Properties – Controlling settings with properties and many other tricks! News Roundup Google Summer of Code 2020: [Final Report] Enhancing Syzkaller support for NetBSD (https://blog.netbsd.org/tnf/entry/google_summer_of_code_20202) Sys2syz would give an extra edge to Syzkaller for NetBSD. It has a potential of efficiently automating the conversion of syscall definitions to syzkaller’s grammar. This can aid in increasing the number of syscalls covered by Syzkaller significantly with the minimum possibility of manual errors. Let’s delve into its internals. How the OpenBSD -stable packages are built (https://dataswamp.org/~solene/2020-10-29-official-openbsd-stable-architecture.html) In this long blog post, I will write about the technical details of the OpenBSD stable packages building infrastructure. I have setup the infrastructure with the help of Theo De Raadt who provided me the hardware in summer 2019, since then, OpenBSD users can upgrade their packages using pkg_add -u for critical updates that has been backported by the contributors. Many thanks to them, without their work there would be no packages to build. Thanks to pea@ who is my backup for operating this infrastructure in case something happens to me. OPNsense 20.7.4 released (https://opnsense.org/opnsense-20-7-4-released/) This release finally wraps up the recent Netmap kernel changes and tests. The Realtek vendor driver was updated as well as third party software cURL, libxml2, OpenSSL, PHP, Suricata, Syslog-ng and Unbound just to name a couple of them. Beastie Bits Binutils and linker changes (https://www.dragonflydigest.com/2020/11/03/25120.html) 28 Years of NetBSD contributions (https://github.com/NetBSD/src/graphs/contributors) Bluetooth Audio on OpenBSD (https://ifconfig.se/bluetooth-audio-openbsd.html) K8s Bhyve (https://k8s-bhyve.convectix.com) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Sean - C Flags (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/376/feedback/Sean%20-%20C%20Flags.md) Thierry - RPI ZFS question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/376/feedback/Thierry%20-%20RPI%20ZFS%20question.md) Thierry's script (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/376/feedback/script.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 12 Nov 2020 03:00:00 -0800
375: Virtually everything
bhyve - The FreeBSD Hypervisor, udf information leak, being a vim user instead of classic vi, FreeBSD on ESXi ARM Fling: Fixing Virtual Hardware, new FreeBSD Remote Process Plugin in LLDB, OpenBSD Laptop, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines bhyve - The FreeBSD Hypervisor (https://klarasystems.com/articles/bhyve-the-freebsd-hypervisor/) FreeBSD has had varying degrees of support as a hypervisor host throughout its history. For a time during the mid-2000s, VMWare Workstation 3.x could be made to run under FreeBSD’s Linux Emulation, and Qemu was ported in 2004, and later the kQemu accelerator in 2005. Then in 2009 a port for VirtualBox was introduced. All of these solutions suffered from being a solution designed for a different operating system and then ported to FreeBSD, requiring constant maintenance. ZFS and FreeBSD Support Klara offers flexible Support Subscriptions for your ZFS and FreeBSD infrastructure. Get a world class team of experts to back you up. Check it out on our website! (https://klarasystems.com/support/) udf info leak (https://gist.github.com/CTurt/a00fb4164e13342567830b052aaed94b) FreeBSD UDF driver info leak Analysis done on FreeBSD release 11.0 because that's what I had around. + Fix committed to FreeBSD (https://svnweb.freebsd.org/changeset/base/366005) News Roundup I'm now a user of Vim, not classical Vi (partly because of windows) (https://utcc.utoronto.ca/~cks/space/blog/unix/VimNowAUser) In the past I've written entries (such as this one) where I said that I was pretty much a Vi user, not really a Vim user, because I almost entirely stuck to Vi features. In a comment on my entry on not using and exploring Vim features, rjc reinforced this, saying that I seemed to be using vi instead of vim (and that there was nothing wrong with this). For a long time I thought this way myself, but these days this is not true any more. These days I really want Vim, not classical Vi. FreeBSD on ESXi ARM Fling: Fixing Virtual Hardware (https://vincerants.com/freebsd-on-esxi-arm-fling-fixing-virtual-hardware/) With the current state of FreeBSD on ARM in general, a number of hardware drivers are either set to not auto-load on boot, or are entirely missing altogether. This page is to document my findings with various bits of hardware, and if possible, list fixes. Introduction of a new FreeBSD Remote Process Plugin in LLDB (https://www.moritz.systems/blog/introduction-of-a-new-freebsd-remote-process-plugin-in-lldb/) Moritz Systems have been contracted by the FreeBSD Foundation to modernize the LLDB debugger’s support for FreeBSD. We are writing a new plugin utilizing the more modern client-server layout that is already used by Darwin, Linux, NetBSD and (unofficially) OpenBSD. The new plugin is going to gradually replace the legacy one. OpenBSD Laptop (https://functionallyparanoid.com/2020/10/14/openbsd-laptop/) Hi, I know it’s been a while. I recently had to nuke and re-pave my personal laptop and I thought it would be a nice thing to share with the community how I set up OpenBSD on it so that I have a useful, modern, secure environment for getting work done. I’m not going to say I’m the expert on this or that this is the BEST way to set up OpenBSD, but I thought it would be worthwhile for folks doing Google searches to at least get my opinion on this. So, given that, let’s go… Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Ethan - Linux user wanting to try out OpenBSD (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/375/feedback/Ethan%20-%20Linux%20user%20wanting%20to%20try%20out%20OpenBSD.md) iian - Learning IT (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/375/feedback/iian%20-%20Learning%20IT.md) johnny - bsd swag (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/375/feedback/johnny%20-%20bsd%20swag.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 05 Nov 2020 03:00:00 -0800
374: OpenBSD’s 25th anniversary
OpenBSD 6.8 has been released, NetBSD 9.1 is out, OpenZFS devsummit report, BastilleBSD’s native container management for FreeBSD, cleaning up old tarsnap backups, Michael W. Lucas’ book sale, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines OpenBSD 6.8 (https://www.openbsd.org/68.html) Released Oct 18, 2020. (OpenBSD's 25th anniversary) NetBSD 9.1 Released (https://www.netbsd.org/releases/formal-9/NetBSD-9.1.html) The NetBSD Project is pleased to announce NetBSD 9.1, the first update of the NetBSD 9 release branch. It represents a selected subset of fixes deemed important for security or stability reasons, as well as new features and enhancements. OpenZFS Developer Summit 2020 (https://klarasystems.com/articles/openzfs-developer-summit-part-1/) As with most other conferences in the last six months, this year’s OpenZFS Developer’s Summit was a bit different than usual. Held via Zoom to accommodate for 2020’s new normal in terms of social engagements, the conference featured a mix of talks delivered live via webinars, and breakout sessions held as regular meetings. This helped recapture some of the “hallway track” that would be lost in an online conference. • After attending the conference, I wrote up some of my notes from each of the talks • Part 2 (https://klarasystems.com/articles/openzfs-developer-summit-part-2/) ZFS and FreeBSD Support Klara offers flexible Support Subscriptions for your ZFS and FreeBSD infrastructure, simply sign up for our monthly subscription! What's even better is that for the month of October we are giving away 3 months for free, for every yearly subscription, and one month free when you sign up for a 6-months subscription! Check it out on our website! (https://klarasystems.com/support/) News Roundup BastilleBSD - native container management for FreeBSD (https://fibric.hashnode.dev/bastillebsd-native-container-management-for-freebsd) Some time ago, I had the requirement to use FreeBSD in a project, and soon the question came up if Docker and Kubernetes can be used. On FreeBSD, Docker is not very well supported, and even if you can get it running, Linux is used in a Docker container. My experience with Docker on FreeBSD is awful, and so I started looking for alternatives. A quick search on one of the most significant online search engines led me to Jails and then to BastilleBSD. Tarsnap – cleaning up old backups (https://dan.langille.org/2020/09/10/tarsnap-cleaning-up-old-backups/) I use Tarsnap for my critical data. Case in point, I use it to backup my Bacula database dump. I use Bacula to backup my hosts. The database in question keeps track of what was backed up, from what host, the file size, checksum, where that backup is now, and many other items. Losing this data is annoying but not a disaster. It can be recreated from the backup volumes, but that is time consuming. As it is, the file is dumped daily, and rsynced to multiple locations. MWL - BookSale (https://mwl.io/archives/8009) For those interested in such things, I recently posted my 60,000th tweet. This prodded me to try an experiment I’ve been pondering for a while. Over at my ebookstore, two of my books are now on a “Name Your Own Price” sale. You can get git commit murder and PAM Mastery for any price you wish, with a minimum of $1. Beastie Bits Brian Kernighan: UNIX, C, AWK, AMPL, and Go Programming | Lex Fridman Podcast #109 (https://www.youtube.com/watch?v=O9upVbGSBFo) The UNIX Time-Sharing System - Dennis M. Ritchie and Ken Thompson - July 1974 (https://chsasank.github.io/classic_papers/unix-time-sharing-system.html#) Using a 1930 Teletype as a Linux Terminal (https://www.youtube.com/watch?v=2XLZ4Z8LpEE) *** ###Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions lars - infosec handbook (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/374/feedback/lars%20-%20infosec%20handbook.md) scott - zfs import (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/374/feedback/scott%20-%20zfs%20import.md) zhong - first episode (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/374/feedback/zhong%20-%20first%20episode.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 29 Oct 2020 04:00:00 -0700
373: Kyle Evans Interview
We have an interview with Kyle Evans for you this week. We talk about his grep project, lua and flua in base, as well as bectl, being on the core team and a whole lot of other stuff. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Interview - Kyle Evans - kevans@freebsd.org (mailto:kevans@freebsd.org) / @kaevans91 (https://twitter.com/kaevans91) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 22 Oct 2020 04:00:00 -0700
372: Slow SSD scrubs
Wayland on BSD, My BSD sucks less than yours, Even on SSDs, ongoing activity can slow down ZFS scrubs drastically, OpenBSD on the Desktop, simple shell status bar for OpenBSD and cwm, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines Wayland on BSD (https://blog.netbsd.org/tnf/entry/wayland_on_netbsd_trials_and) After I posted about the new default window manager in NetBSD I got a few questions, including "when is NetBSD switching from X11 to Wayland?", Wayland being X11's "new" rival. In this blog post, hopefully I can explain why we aren't yet! My BSD sucks less than yours (https://www.bsdfrog.org/pub/events/my_bsd_sucks_less_than_yours-full_paper.pdf) This paper will look at some of the differences between the FreeBSD and OpenBSD operating systems. It is not intended to be solely technical but will also show the different "visions" and design decisions that rule the way things are implemented. It is expected to be a subjective view from two BSD developers and does not pretend to represent these projects in any way. Video + EuroBSDCon 2017 Part 1 (https://www.youtube.com/watch?v=ZhpaKuXKob4) + EuroBSDCon 2017 Part 2 (https://www.youtube.com/watch?v=cYp70KWD824) News Roundup Even on SSDs, ongoing activity can slow down ZFS scrubs drastically (https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSSSDActivitySlowsScrubs) Back in the days of our OmniOS fileservers, which used HDs (spinning rust) across iSCSI, we wound up changing kernel tunables to speed up ZFS scrubs and saw a significant improvement. When we migrated to our current Linux fileservers with SSDs, I didn't bother including these tunables (or the Linux equivalent), because I expected that SSDs were fast enough that it didn't matter. Indeed, our SSD pools generally scrub like lightning. OpenBSD on the Desktop (Part I) (https://paedubucher.ch/articles/2020-09-05-openbsd-on-the-desktop-part-i.html) Let's install OpenBSD on a Lenovo Thinkpad X270. I used this computer for my computer science studies. It has both Arch Linux and Windows 10 installed as dual boot. Now that I'm no longer required to run Windows, I can ditch the dual boot and install an operating system of my choice. A simple shell status bar for OpenBSD and cwm(1) (https://www.tumfatig.net/20200923/a-simple-shell-status-bar-for-cwm/) These days, I try to use simple and stock software as much as possible on my OpenBSD laptop. I’ve been playing with cwm(1) for weeks and I was missing a status bar. After trying things like Tint2, Polybar etc, I discovered @gonzalo’s termbar. Thanks a lot! As I love scripting, I decided to build my own. Beastie Bits DragonFly v5.8.3 released to address to issues (http://lists.dragonflybsd.org/pipermail/commits/2020-September/769777.html) OpenSSH 8.4 released (http://www.openssh.com/txt/release-8.4) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Dane - FreeBSD vs Linux in Microservices and Containters (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/372/feedback/Dane%20-%20FreeBSD%20vs%20Linux%20in%20Microservices%20and%20Containters.md) Mason - questions.md (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/372/feedback/Mason%20-%20questions.md) Michael - Tmux License.md (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/372/feedback/Michael%20-%20Tmux%20License.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 15 Oct 2020 03:00:00 -0700
371: Wildcards running wild
New Project: zedfs.com, TrueNAS CORE Ready for Deployment, IPC in FreeBSD 11: Performance Analysis, Unix Wildcards Gone Wild, Unix Wars, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines My New Project: zedfs.com (https://www.oshogbo.vexillium.org/blog/80/) Have you ever had an idea that keeps coming back to you over and over again? For a week? For a month? I know that feeling. My new project was born from this feeling. On this blog, I mix content a lot. I have written personal posts (not many of them, but still), FreeBSD development posts, development posts, security posts, and ZFS posts. This mixed content can be problematic sometimes. I share a lot of stuff here, and readers don’t know what to expect next. I am just excited by so many things, and I want to share that excitement with you! TrueNAS CORE is Ready for Deployment (https://www.ixsystems.com/blog/truenas-12-rc-1/) TrueNAS 12.0 RC1 was released yesterday and with it, TrueNAS CORE is ready for deployment. The merger of FreeNAS and TrueNAS into a unified software image can now begin its path into mainstream use. TrueNAS CORE is the new FreeNAS and is on schedule. The TrueNAS 12.0 BETA process started in June and has been the most successful BETA release ever with more than 3,000 users and only minor issues. Ars Technica provided a detailed technical walkthrough of the original BETA. There is a long list of features and performance improvements. During the BETA process, TrueNAS 12.0 demonstrated over 1.2 Million IOPS and over 23GB/s on a TrueNAS M60. News Roundup Interprocess Communication in FreeBSD 11: Performance Analysis (https://arxiv.org/pdf/2008.02145.pdf) Interprocess communication, IPC, is one of the most fundamental functions of a modern operating system, playing an essential role in the fabric of contemporary applications. This report conducts an investigation in FreeBSD of the real world performance considerations behind two of the most common IPC mechanisms; pipes and sockets. A simple benchmark provides a fair sense of effective bandwidth for each, and analysis using DTrace, hardware performance counters and the operating system’s source code is presented. We note that pipes outperform sockets by 63% on average across all configurations, and further that the size of userspace transmission buffers has a profound effect on performance — larger buffers are beneficial up to a point (∼ 32-64 KiB) after which performance collapses as a result of devastating cache exhaustion. A deep scrutiny of the probe effects at play is also presented, justifying the validity of conclusions drawn from these experiments. Back To The Future: Unix Wildcards Gone Wild (https://www.defensecode.com/public/DefenseCode_Unix_WildCards_Gone_Wild.txt) First of all, this article has nothing to do with modern hacking techniques like ASLR bypass, ROP exploits, 0day remote kernel exploits or Chrome's Chain-14-Different-Bugs-To-Get-There... Nope, nothing of the above. This article will cover one interesting old-school Unix hacking technique, that will still work nowadays in 2013. Unix Wars (https://www.livinginternet.com/i/iw_unix_war.htm) Dozens of different operating systems have been developed over the years, but only Unix has grown in so many varieties. There are three main branches. Four factors have facilitated this growth... Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Chris - installing FreeBSD 13-current (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/371/feedback/Chris%20-%20installing%20FreeBSD%2013-current.md) Dane - FreeBSD History Lesson (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/371/feedback/Dane%20-%20FreeBSD%20History%20Lesson.md) Marc - linux compat (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/371/feedback/Marc%20-%20linux%20compat.md) Mason - apropos battery (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/371/feedback/Mason%20-%20apropos%20battery.md) Paul - a topic idea (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/371/feedback/Paul%20-%20a%20topic%20idea.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 08 Oct 2020 03:00:00 -0700
370: Testing shutdown
The world’s first OpenZFS based live image, FreeBSD Subversion to Git Migration video, FreeBSD Instant-workstation 2020, testing the shutdown mechanism, login_ldap added to OpenBSD, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines FuryBSD 2020-Q3 The world’s first OpenZFS based live image (https://www.furybsd.org/furybsd-2020-q3-the-worlds-first-openzfs-based-live-image/) FuryBSD is a tool to test drive stock FreeBSD desktop images in read write mode to see if it will work for you before installing. In order to provide the most reliable experience possible while preserving the integrity of the system the LiveCD now leverages ZFS, compression, replication, a memory file system, and reroot (pivot root). FreeBSD Subversion to Git Migration: Pt 1 Why? (https://bsdimp.blogspot.com/2020/09/freebsd-subversion-to-git-migration.html) FreeBSD moving to Git: Why? With luck, I'll be writing a few blogs on FreeBSD's move to git later this year. Today, we'll start with "why"? Video from Warner Losh (https://www.youtube.com/watch?v=Lx9lKr_M-DI) News Roundup FreeBSD Instant-workstation 2020 (https://euroquis.nl/freebsd/2020/09/17/instant-workstation.html) A little over a year ago I published an instant-workstation script for FreeBSD. The idea is to have an installed FreeBSD system, then run a shell script that uses only base-system utilities and installs and configures a workstation setup for you. nut – testing the shutdown mechanism (https://dan.langille.org/2020/09/10/nut-testing-the-shutdown-mechanism/) Following on from my recent nut setup, this is the second in a series of three posts. The next post will deal with adjusting startup and shutdown times to be sure everything proceeds as required. login_ldap added to OpenBSD -current (https://undeadly.org/cgi?action=article;sid=20200913081040) With this commit, Martijn van Duren (martijn@) added login_ldap(8) to -current + https://marc.info/?l=openbsd-cvs&m=159992319027593&w=2 Beastie Bits NetBSD current now has GCC 9.3.0 for x86/ARM (https://twitter.com/netbsd/status/1305082782457245696) MidnightBSD 1.2.8 (https://www.justjournal.com/users/mbsd/entry/33802) MidnightBSD 2.0-Current (https://www.justjournal.com/users/mbsd/entry/33806) Retro UNIX 8086 v1 operating system has been developed by Erdogan Tan as a special purposed derivation of original UNIX v1 (https://www.singlix.com/runix/) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Rick - rcorder (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/370/feedback/Rick%20-%20rcorder.md) Dan - machiatto bin (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/370/feedback/dan%20-%20machiatto%20bin.md) Luis - old episodes (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/370/feedback/luis%20-%20old%20episodes.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 01 Oct 2020 03:15:00 -0700
369: Where rc.d belongs
High Availability Router/Firewall Using OpenBSD, CARP, pfsync, and ifstated, Building the Development Version of Emacs on NetBSD, rc.d belongs in libexec, not etc, FreeBSD 11.3 EOL, OPNsense 20.7.1 Released, MidnightBSD 1.2.7 out, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) Headlines High Availability Router/Firewall Using OpenBSD, CARP, pfsync, and ifstated (https://dzone.com/articles/high-availability-routerfirewall-using-openbsd-car) I have been running OpenBSD on a Soekris net5501 for my router/firewall since early 2012. Because I run a multitude of services on this system (more on that later), the meager 500Mhz AMD Geode + 512MB SDRAM was starting to get a little sluggish while trying to do anything via the terminal. Despite the perceived performance hit during interactive SSH sessions, it still supported a full 100Mbit connection with NAT, so I wasn’t overly eager to change anything. Luckily though, my ISP increased the bandwidth available on my plan tier to 150Mbit+. Unfortunately, the Soekris only contained 4xVIA Rhine Fast Ethernet. So now, I was using a slow system and wasting money by not being able to fully utilize my connection. Building the Development Version of Emacs on NetBSD (https://lars.ingebrigtsen.no/2020/08/25/building-the-development-version-of-emacs-on-netbsd/) I hadn’t really planned on installing a NetBSD VM (after doing all the other two BSDs), but then a NetBSD-related Emacs bug report arrived. News Roundup rc.d belongs in libexec, not etc (https://jmmv.dev/2020/08/rcd-libexec-etc.html) Let’s open with the controversy: the scripts that live under /etc/rc.d/ in FreeBSD, NetBSD, and OpenBSD are in the wrong place. They all should live in /libexec/rc.d/ because they are code, not configuration. This misplacement is something that has bugged me for ages but I never had the energy to open this can of worms back when I was very involved in NetBSD. I suspect it would have been a draining discussion and a very difficult thing to change. FreeBSD 11.3 EOL (https://lists.freebsd.org/pipermail/freebsd-announce/2020-September/001982.html) As of September 30, 2020, FreeBSD 11.3 will reach end-of-life and will no longer be supported by the FreeBSD Security Team. Users of FreeBSD 11.3 are strongly encouraged to upgrade to a newer release as soon as possible. OPNsense 20.7.1 Released (https://opnsense.org/opnsense-20-7-1-released/) Overall, the jump to HardenedBSD 12.1 is looking promising from our end. From the reported issues we still have more logging quirks to investigate and especially Netmap support (used in IPS and Sensei) is lacking in some areas that were previously working. Patches are being worked on already so we shall get there soon enough. Stay tuned. MidnightBSD 1.2.7 out (https://www.justjournal.com/users/mbsd/entry/33801) MidnightBSD 1.2.7 is available via the FTP/HTTP and mirrors as well as github. It includes several bug fixes and security updates over the last ISO release and is recommended for new installations. Users who don't want to updatee the whole OS, should consider at least updating libmport as there are many package management fixes Beastie Bits Tarsnap podcast (https://blog.firosolutions.com/2020/08/tarsnap-podcast/) NetBSD Tips and Tricks (http://students.engr.scu.edu/~sschaeck/netbsd/index.html) FreeBSD mini-git Primer (https://hackmd.io/hJgnfzd5TMK-VHgUzshA2g) GhostBSD Financial Reports (https://ghostbsd.org/financial_reports_from_January_to_June_2020) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Daniel - Documentation Tooling (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/369/feedback/Daniel%20-%20Documentation%20Tooling.md) Fongaboo - Where did the ZFS tutorial Go? (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/369/feedback/Fongaboo%20-%20Where%20did%20the%20ZFS%20Tutorial%20Go.md) Johnny - Browser Cold Wars (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/369/feedback/Johnny%20-%20Browser%20Cold%20Wars.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 24 Sep 2020 09:00:00 -0700
368: Changing OS roles
Modernizing the OpenBSD Console, OS roles have changed, FreeBSD Cluster with Pacemaker and Corosync, Wine in a 32-bit sandbox on 64-bit NetBSD, Find package which provides a file in OpenBSD, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines Modernizing the OpenBSD Console (https://www.cambus.net/modernizing-the-openbsd-console/) At the beginning were text mode consoles. Traditionally, *BSD and Linux on i386 and amd64 used text mode consoles which by default provided 25 rows of 80 columns, the "80x25 mode". This mode uses a 8x16 font stored in the VGA BIOS (which can be slightly different across vendors). OpenBSD uses the wscons(4) console framework, inherited from NetBSD OS roles have changed (https://rubenerd.com/the-roles-of-oss-have-changed/) Though I do wonder sometimes, with just a slight tweak to history, how things might have been different. In another dimension somewhere, I’m using the latest BeOS-powered PowerPC laptop, and a shiny new Palm smartphone. Both of these represented the pinnacle of UI design in the 1990s, and still in the 2020s have yet to be surpassed. People call me an Apple fanboy, but I’d drop all of it in a second for that gear. News Roundup FreeBSD Cluster with Pacemaker and Corosync (https://vermaden.wordpress.com/2020/09/03/freebsd-cluster-with-pacemaker-and-corosync/) I always missed ‘proper’ cluster software for FreeBSD systems. Recently I got to run several Pacemaker/Corosync based clusters on Linux systems. I thought how to make similar high availability solutions on FreeBSD and I was really shocked when I figured out that both Pacemaker and Corosync tools are available in the FreeBSD Ports and packages as net/pacemaker2 and net/corosync2 respectively. Wine in a 32-bit sandbox on 64-bit NetBSD (https://washbear.neocities.org/wine-sandbox.html) "Mainline pkgsrc" can't do strange multi-arch Wine builds yet, so a 32-bit sandbox seems like a reasonable way to use 32-bit Wine on amd64 without resorting to running real Windows in NVMM. We'll see if this was a viable alternative to re-reviewing the multi-arch support in pkgsrc-wip... We're using sandboxctl, which is a neat tool for quickly shelling into a different NetBSD userspace. Maybe you also don't trust the Windows applications you're running too much - sandboxctl creates a chroot based on a fresh system image, and chroot on NetBSD is fairly bombproof. Find package which provides a file in OpenBSD (https://dataswamp.org/~solene/2020-09-04-pkglocate-openbsd.html) There is one very handy package on OpenBSD named pkglocatedb which provides the command pkglocate. If you need to find a file or binary/program and you don’t know which package contains it, use pkglocate. Beastie Bits OpenBSD for 1.5 Years: Confessions of a Linux Heretic (https://www.youtube.com/watch?v=oTShQIXSdqM) OpenBSD 6.8 Beta Tagged (https://undeadly.org/cgi?action=article;sid=20200831192811) Hammer2 and growth (https://www.dragonflydigest.com/2020/09/08/24933.html) Understanding a FreeBSD kernel vulnerability (https://www.thezdi.com/blog/2020/9/1/cve-2020-7460-freebsd-kernel-privilege-escalation) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Rob - 7 years (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/368/feedback/Bruce%20-%207%20years.md) Kurt - Microserver (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/368/feedback/Kurt%20-%20Microserver.md) Rob - Interviews (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/368/feedback/Rob%20-%20Interviews.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 17 Sep 2020 03:00:00 -0700
367: Changing jail datasets
A 35 Year Old Bug in Patch, Sandbox for FreeBSD, Changing from one dataset to another within a jail, You don’t need tmux or screen for ZFS, HardenedBSD August 2020 Status Report and Call for Donations, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines A 35 Year Old Bug in Patch (http://bsdimp.blogspot.com/2020/08/a-35-year-old-bug-in-patch-found-in.html) Larry Wall posted patch 1.3 to mod.sources on May 8, 1985. A number of versions followed over the years. It's been a faithful alley for a long, long time. I've never had a problem with patch until I embarked on the 2.11BSD restoration project. In going over the logs very carefully, I've discovered a bug that bites this effort twice. It's quite interesting to use 27 year old patches to find this bug while restoring a 29 year old OS... Sandbox for FreeBSD (https://www.relkom.sk/en/fbsd_sandbox.shtml) A sandbox is a software which artificially limits access to the specific resources on the target according to the assigned policy. The sandbox installs hooks to the kernel syscalls and other sub-systems in order to interrupt the events triggered by the application. From the application point of view, application working as usual, but when it wants to access, for instance, /dev/kmem the sandbox software decides against the assigned sandbox scheme whether to grant or deny access. In our case, the sandbox is a kernel module which uses MAC (Mandatory Access Control) Framework developed by the TrustedBSD team. All necessary hooks were introduced to the FreeBSD kernel. Source Code (https://gitlab.com/relkom/sandbox) Documentation (https://www.relkom.sk/en/fbsd_sandbox_docs.shtml) News Roundup Changing from one dataset to another within a jail (https://dan.langille.org/2020/08/16/changing-from-one-dataset-to-another-within-a-freebsd-iocage-jail/) ZFS has a the ability to share itself within a jail. That gives the jail some autonomy, and I like that. I’ve written briefly about that, specifically for iocage. More recently, I started using a zfs snapshot for caching clearing. The purpose of this post is to document the existing configuration of the production FreshPorts webserver and outline the plan on how to modify it for more zfs-snapshot-based cache clearing. You don’t need tmux or screen for ZFS (https://rubenerd.com/you-dont-need-tmux-or-screen-for-zfs/) Back in January I mentioned how to add redundancy to a ZFS pool by adding a mirrored drive. Someone with a private account on Twitter asked me why FreeBSD—and NetBSD!—doesn’t ship with a tmux or screen equivilent in base in order to daemonise the process and let them run in the background. ZFS already does this for its internal commands. HardenedBSD August 2020 Status Report and Call for Donations (https://hardenedbsd.org/article/shawn-webb/2020-08-15/hardenedbsd-august-2020-status-report-and-call-donations) This last month has largely been a quiet one. I've restarted work on porting five-year-old work from the Code Pointer Integrity (CPI) project into HardenedBSD. Chiefly, I've started forward-porting the libc and rtld bits from the CPI project and now need to look at llvm compiler/linker enhancements. We need to be able to apply SafeStack to shared objects, not just application binaries. This forward-porting work I'm doing is to support that effort. The infrastructure has settled and is now churning normally and happily. We're still working out bandwidth issues. We hope to have a new fiber line ran by the end of September. As part of this status report, I'm issuing a formal call for donations. I'm aiming for $4,000.00 USD for a newer self-hosted Gitea server. I hope to purchase the new server before the end of 2020. Important parts of Unix's history happened before readline support was common (https://utcc.utoronto.ca/~cks/space/blog/unix/TimeBeforeReadline) Unix and things that run on Unix have been around for a long time now. In particular, GNU Readline was first released in 1989 (as was Bash), which is long enough ago for it (or lookalikes) to become pretty much pervasive, especially in Unix shells. Today it's easy to think of readline support as something that's always been there. But of course this isn't the case. Unix in its modern form dates from V7 in 1979 and 4.2 BSD in 1983, so a lot of Unix was developed before readline and was to some degree shaped by the lack of it. Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Mason - mailserver (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/367/feedback/Mason%20-%20mailserver.md) casey - freebsd on decline (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/367/feedback/casey%20-%20freebsd%20on%20decline.md) denis - postgres (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/367/feedback/denis%20-%20postgres.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 10 Sep 2020 03:00:00 -0700
366: Bootloader zpool checkpoints
OpenZFS with ZSTD lands in FreeBSD 13, LibreSSL doc status update, FreeBSD on SPARC64 (is dead), Bringing zpool checkpoints to a FreeBSD bootloader, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines OpenZFS with ZSTD land in FreeBSD 13 (https://svnweb.freebsd.org/base?view=revision&revision=364746) ZStandard Compression for OpenZFS (https://github.com/openzfs/zfs/commit/10b3c7f5e424f54b3ba82dbf1600d866e64ec0a0) > The primary benefit is maintaining a completely shared code base with the community allowing FreeBSD to receive new features sooner and with less effort. > I would advise against doing 'zpool upgrade' or creating indispensable pools using new features until this change has had a month+ to soak. Rebasing FreeBSD’s OpenZFS on the new upstream was sponsored by iXsystems The competition of ZSTD support for OpenZFS was sponsored by the FreeBSD Foundation *** LibreSSL documentation status update (https://undeadly.org/cgi?action=article;sid=20200817063735) More than six years ago, LibreSSL was forked from OpenSSL, and almost two years ago, i explained the status of LibreSSL documentation during EuroBSDCon 2018 in Bucuresti. So it seems providing an update might be in order. Note that this is not an update regarding LibreSSL status in general because i'm not the right person to talk about the big picture of working on the LibreSSL code, my work has been quite focussed on documentation. All the same, it is fair to say that even though the number of developers working on it is somewhat limited, the LibreSSL project is quite alive, typically having a release every few months. Progress continues being made with respect to porting and adding new functionality (for example regarding TLSv1.3, CMS, RSA-PSS, RSA-OAEP, GOST, SM3, SM4, XChaCha20 during the last two years), OpenSSL compatibility improvements (including providing additional OpenSSL-1.1 APIs), and lots of bug fixes and code cleanup. FreeBSD on SPARC64 (is dead) (https://eerielinux.wordpress.com/2020/02/15/freebsd-on-sparc64-is-dead/) ’m coming pretty late to the party, because SPARC64 support in FreeBSD is apparently doomed: After the POWER platform made the switch to a LLVM/Clang-based toolchain, SPARC64 is one of the last ones that still uses the ancient GCC 4.2-based toolchain that the project wants to finally get rid off (it has already happened as I was writing this – looks like the firm plan was not so firm after all, since they killed it off early). And compared to the other platforms it has seen not too much love in recent times… SPARC64 being a great platform, I’d be quite sad to see it go. But before that happens let’s see what the current status is and what would need to be done if it were to survive, shall we? News Roundup Bringing zpool checkpoints to a FreeBSD bootloader (https://www.oshogbo.vexillium.org/blog/79/) Almost two years ago I wrote a blog post about checkpoints in ZFS. I didn’t hide that I was a big fan of them. That said, after those two years, I still feel that there are underappreciated features in the ZFS world, so I decided to do something about that. Currently, one of the best practices for upgrading your operating system is to use boot environments. They are a great feature for managing multiple kernels and userlands. They are based on juggling which ZFS datasets are mounted. Each dataset has its own version of the system. Unfortunately, boot environments have their limitations. If we, for example, upgrade our ZFS pool, we may not be able to use older versions of the system anymore. The big advantage of boot environments is that they have very good tools. Two main tools are beadm (which was created by vermaden) and bectl (which currently is in the FreeBSD base system). These tools allow us to create and manage boot environments. Beastie Bits The First Unix Port (https://documents.uow.edu.au/content/groups/public/@web/@inf/@scsse/documents/doc/uow103747.pdf) TLS Mastery updates, August 2020 (https://mwl.io/archives/7346) What is the Oldest BSD Distribution still around today (https://www.youtube.com/watch?v=ww60o940kEk) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions ben - zfs send questions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/366/feedback/ben%20-%20zfs%20send%20questions.md) lars - zfs pool question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/366/feedback/lars%20-%20zfs%20pool%20question.md) neutron - bectl vs beadm (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/366/feedback/neutron%20-%20bectl%20vs%20beadm.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 03 Sep 2020 03:00:00 -0700
365: Whole year round
FreeBSD USB Audio, Kyua: An introduction for NetBSD users, Keeping backup ZFS on Linux kernel modules around, CLI Tools 235x Faster than Hadoop, FreeBSD Laptop Battery Life Status Command, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines FreeBSD USB Audio (https://www.davidschlachter.com/misc/freebsd-usb-audio) I recently got a Behringer UMC22 sound card for video conferencing and DJing. This page documents what I’ve learned about using this sound card, and USB audio in general, on FreeBSD. tl;dr: Everything works as long as the sound card follows the USB audio device class specification. Kyua: An introduction for NetBSD users (https://wiki.netbsd.org/kyua/) Kyua's current goal is to reimplement only the ATF tools while maintaining backwards compatibility with the tests written with the ATF libraries (i.e. with the NetBSD test suite). Because Kyua is a replacement of some ATF components, the end goal is to integrate Kyua into the NetBSD base system (just as ATF is) and remove the deprecated ATF components. Removing the deprecated components will allow us to make the above-mentioned improvements to Kyua, as well as many others, without having to deal with the obsolete ATF code base. Discussing how and when this transition might happen is out of the scope of this document at the moment. News Roundup Keeping backup ZFS on Linux kernel modules around (https://utcc.utoronto.ca/~cks/space/blog/linux/ZFSOnLinuxModuleBackups) I'm a long term user of ZFS on Linux and over pretty much all of the time I've used it, I've built it from the latest development version. Generally this means I update my ZoL build at the same time as I update my Fedora kernel, since a ZoL update requires a kernel reboot anyway. This is a little bit daring, of course, although the ZoL development version has generally been quite solid (and this way I get the latest features and improvements long before I otherwise would). Command-line Tools can be 235x Faster than your Hadoop Cluster (https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html) As I was browsing the web and catching up on some sites I visit periodically, I found a cool article from Tom Hayden about using Amazon Elastic Map Reduce (EMR) and mrjob in order to compute some statistics on win/loss ratios for chess games he downloaded from the millionbase archive, and generally have fun with EMR. Since the data volume was only about 1.75GB containing around 2 million chess games, I was skeptical of using Hadoop for the task, but I can understand his goal of learning and having fun with mrjob and EMR. Since the problem is basically just to look at the result lines of each file and aggregate the different results, it seems ideally suited to stream processing with shell commands. I tried this out, and for the same amount of data I was able to use my laptop to get the results in about 12 seconds (processing speed of about 270MB/sec), while the Hadoop processing took about 26 minutes (processing speed of about 1.14MB/sec). FreeBSD Laptop Find Out Battery Life Status Command (https://www.cyberciti.biz/faq/freebsd-finding-out-battery-life-state-on-laptop/) I know how to find out battery life status using Linux operating system. How do I monitor battery status on a laptop running FreeBSD version 9.x/10.x/11.x/12.x? You can use any one of the following commands to get battery status under FreeBSD laptop including remaining battery life and more. Beastie Bits BSD Beer (https://i.redd.it/hlh8luidzgg51.jpg) Awk for JSON (https://github.com/mohd-akram/jawk) Drawing Pictures The Unix Way - with pic and troff (https://youtu.be/oG2A_1vC6aM) Refactoring the FreeBSD Kernel with Checked C (https://www.cs.rochester.edu/u/jzhou41/papers/freebsd_checkedc.pdf) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Jason - German Locales (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/365/jason%20-%20german%20locale.md) pcwizz - Router Style Device (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/365/pcwizz%20-%20router%20style%20device.md) predrag - OpenBSD Router Hardware (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/365/predrag%20-%20openbsd%20router%20hardware.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 27 Aug 2020 04:00:00 -0700
364: FreeBSD Wireless Grind
FreeBSD Qt WebEngine GPU Acceleration, the grind of FreeBSD’s wireless stack, thoughts on overlooking Illumos's syseventadm, when Unix learned to reboot, New EXT2/3/4 File-System driver in DragonflyBSD, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines FreeBSD Qt WebEngine GPU Acceleration (https://euroquis.nl/freebsd/2020/07/21/webengine.html) FreeBSD has a handful of Qt WebEngine-based browsers. Falkon, and Otter-Browser, and qutebrowser and probably others, too. All of them can run into issues on FreeBSD with GPU-accelerated rendering not working. Let’s look at some of the workarounds. NetBSD on the Nanopi Neo2 (https://www.cambus.net/netbsd-on-the-nanopi-neo2/) The NanoPi NEO2 from FriendlyARM has been serving me well since 2018, being my test machine for OpenBSD/arm64 related things. As NetBSD/evbarm finally gained support for AArch64 in NetBSD 9.0, released back in February, I decided to give it a try on this device. The board only has 512MB of RAM, and this is where NetBSD really shines. Things have become a lot easier since jmcneill@ now provides bootable ARM images for a variety of devices, including the NanoPi NEO2. I'm back into the grind of FreeBSD's wireless stack and 802.11ac (https://adrianchadd.blogspot.com/2020/07/im-back-into-grind-of-freebsds-wireless.html) Yes, it's been a while since I posted here and yes, it's been a while since I was actively working on FreeBSD's wireless stack. Life's been .. well, life. I started the ath10k port in 2015. I wasn't expecting it to take 5 years, but here we are. My life has changed quite a lot since 2015 and a lot of the things I was doing in 2015 just stopped being fun for a while. But the stars have aligned and it's fun again, so here I am. News Roundup Some thoughts on us overlooking Illumos's syseventadm (https://utcc.utoronto.ca/~cks/space/blog/solaris/OverlookingSyseventadm) In a comment on my praise of ZFS on Linux's ZFS event daemon, Joshua M. Clulow noted that Illumos (and thus OmniOS) has an equivalent in syseventadm, which dates back to Solaris. I hadn't previously known about syseventadm, despite having run Solaris fileservers and OmniOS fileservers for the better part of a decade, and that gives me some tangled feelings. When Unix learned to reboot (https://bsdimp.blogspot.com/2020/07/when-unix-learned-to-reboot2.html) Recently, a friend asked me the history of halt, and when did we have to stop with the sync / sync / sync dance before running halt or reboot. The two are related, it turns out. DragonFlyBSD Lands New EXT2/3/4 File-System Driver (https://www.phoronix.com/scan.php?page=news_item&px=DragonFlyBSD-New-EXT2FS) While DragonFlyBSD has its own, original HAMMER2 file-system, for those needing to access data from EXT2/EXT3/EXT4 file-systems, there is a brand new "ext2fs" driver implementation for this BSD operating system. DragonFlyBSD has long offered an EXT2 file-system driver (that also handles EXT3 and EXT4) while hitting their Git tree this week is a new version. The new sys/vfs/ext2fs driver, which will ultimately replace their existing sys/gnu/vfs/ext2fs driver is based on a port from FreeBSD code. As such, this driver is BSD licensed rather than GPL. But besides the more liberal license to jive with the BSD world, this new driver has various feature/functionality improvements over the prior version. However, there are some known bugs so for the time being both file-system drivers will co-exist. Beastie Bits LibreOffice 7.0 call for testing (https://lists.freebsd.org/pipermail/freebsd-office/2020-July/005822.html) More touchpad support (https://www.dragonflydigest.com/2020/07/15/24747.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Casey - openbsd wirewall (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/364/feedback/casey%20-%20openbsd%20wirewall.md) Daryl - zfs (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/364/feedback/daryl%20-%20zfs.md) Raymond - hpe microserver (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/364/feedback/raymond%20-%20hpe%20microserver.md) - Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 20 Aug 2020 04:00:00 -0700
363: Traditional Unix toolchains
FreeBSD Q2 Quarterly Status report of 2020, Traditional Unix Toolchains, BastilleBSD 0.7 released, Finding meltdown on DragonflyBSD, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines FreeBSD Quarterly Report (https://www.freebsd.org/news/status/report-2020-04-2020-06.html) This report will be covering FreeBSD related projects between April and June, and covers a diverse set of topics ranging from kernel updates over userland and ports, as well to third-party work. Some highlights picked with the roll of a d100 include, but are not limited to, the ability to forcibly unmounting UFS when the underlying media becomes inaccessible, added preliminary support for Bluetooth Low Energy, a introduction to the FreeBSD Office Hours, and a repository of software collections called potluck to be installed with the pot utility, as well as many many more things. As a little treat, readers can also get a rare report from the quarterly team. Finally, on behalf of the quarterly team, I would like to extend my deepest appreciation and thank you to salvadore@, who decided to take down his shingle. His contributions not just the quarterly reports themselves, but also the surrounding tooling to many-fold ease the work, are immeasurable. Traditional Unix Toolchains (https://bsdimp.blogspot.com/2020/07/traditional-unix-toolchains.html?m=1) Older Unix systems tend to be fairly uniform in how they handle the so-called 'toolchain' for creating binaries. This blog will give a quick overview of the toolchain pipeline for Unix systems that follow the V7 tradition (which evolved along with Unix, a topic for a separate blog maybe). Unix is a pipeline based system, either physically or logically. One program takes input, process the data and produces output. The input and output have some interface they obey, usually text-based. The Unix toolchain is no different. News Roundup Bastille Day 2020 : v0.7 released (https://github.com/BastilleBSD/bastille/releases/tag/0.7.20200714) This release matures the project from 0.6.x -> 0.7.x. Continued testing and bug fixes are proving Bastille capable for a range of use-cases. New (experimental) features are examples of innovation from community contribution and feedback. Thank you. Beastie Bits Finding meltdown on DragonFly (https://www.dragonflydigest.com/2020/07/28/24787.html) NetBSD Server Outage (https://mobile.twitter.com/netbsd/status/1286898183923277829) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Vincent - Gnome 3 question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/363/feedback/vincent%20-%20gnome3.md) Malcolm - ZFS question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/363/feedback/malcolm%20-%20zfs.md) Hassan - Video question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/363/feedback/hassan%20-%20video.md) For those that watch on youtube, don’t forget to subscribe to our new YouTube Channel if you want updates when we post them on YT (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/363/feedback/new-bsdnow-youtube-channel.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 13 Aug 2020 04:00:00 -0700
362: 2.11-BSD restoration
Interview with Warner Losh about Unix history, the 2.11-BSD restoration project, the Unix heritage society, proper booting, and what devmatch is. Interview - Warner Losh - imp@freebsd.org (mailto:imp@freebsd.org) / @bsdimp (https://twitter.com/bsdimp) BSD 2.11 restoration project Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Special Guest: Warner Losh.
Thu, 06 Aug 2020 05:00:00 -0700
361: Function-based MicroVM
Emulex: The Cheapest 10gbe for Your Homelab, In Search of 2.11BSD, as released, Fakecracker: NetBSD as a Function Based MicroVM, First powerpc64 snapshots available for OpenBSD, OPNsense 20.1.8 released, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines Emulex: The Cheapest 10gbe for Your Homelab (https://vincerants.com/emulex-the-cheapest-10gbe/) Years ago, the hunt for the cheapest 10gbe NICs resulted in buying Mellanox ConnectX-2 single-port 10gbe network cards from eBay for around $10. Nowadays those cards have increased in cost to around $20-30. While still cheap, not quite the cheapest. There are now alternatives! Before diving into details, let’s get something very clear. If you want the absolute simplest plug-and-play 10gbe LAN for your homelab, pay the extra for Mellanox. If you’re willing to go hands-on, do some simple manual configuration and installation, read on for my experiences with Emulex 10gbe NICs. Emulex NICs can often be had for around $15 on eBay, sometimes even cheaper. I recently picked up a set of 4 of these cards, which came bundled with 6 SFP+ 10g-SR modules for a grand total of $47.48. Considering I can usually find SFP+ modules for about $5/ea, these alone were worth $30. + I have also tried some Solarflare cards that I found cheap, they work ok, but are pickier about optics, and tend to be focused on low-latency, so often don’t manage to saturate the full 10 gbps, topping out around 8 gbps. + I have been using fs.com for optics, patch cables, and DACs. I find DACs are usually cheaper if you are just going between a server and a switch in the same rack, or direct between 2 servers. In Search of 2.11BSD, as released (https://bsdimp.blogspot.com/2020/07/211bsd-original-tapes-recreation.html) Almost all of the BSD releases have been well preserved. If you want to find 1BSD, or 2BSD or 4.3-TAHOE BSD you can find them online with little fuss. However, if you search for 2.11BSD, you'll find it easily enough, but it won't be the original. You'll find either the latest patched version (2.11BSD pl 469), or one of the earlier popular version (pl 430 is popular). You can even find the RetroBSD project which used 2.11BSD as a starting point to create systems for tiny mips-based PIC controllers. You'll find every single patch that's been issued for the system. News Roundup Fakecracker: NetBSD as a Function Based MicroVM (https://imil.net/blog/posts/2020/fakecracker-netbsd-as-a-function-based-microvm/) In November 2018 AWS published an Open Source tool called Firecracker, mostly a virtual machine monitor relying on KVM, a small sized Linux kernel, and a stripped down version of Qemu. What baffled me was the speed at which the virtual machine would fire up and run the service. The whole process is to be compared to a container, but safer, as it does not share the kernel nor any resource, it is a separate and dedicated virtual machine. If you want to learn more on Firecracker‘s internals, here’s a very well put article. First powerpc64 snapshots available for OpenBSD (https://undeadly.org/cgi?action=article;sid=20200707001113) Since we reported the first bits of powerpc64 support going into the tree on 16 May, work has progressed at a steady pace, resulting in snapshots now being available for this platform. So, if you have a POWER9 system idling around, go to your nearest mirror and fetch this snapshot. Keep in mind that as this is still very early days, very little handholding is available - you are basically on your own. OPNsense 20.1.8 released (https://opnsense.org/opnsense-20-1-8-released/) Sorry about the delay while we chased a race condition in the updates back to an issue with the latest FreeBSD package manager updates. For now we reverted to our current version but all relevant third party packages have been updated as updates became available over the last weeks, e.g. cURL and Python, and hostapd / wpa_supplicant amongst others. Beastie Bits Old School Disk Partitioning (https://bsdimp.blogspot.com/2020/07/old-school-disk-partitioning.html) Nomad BSD 1.3.2 Released (http://nomadbsd.org/index.html#1.3.2) Chai-Fi (https://github.com/gonzoua/chaifi) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Poojan - ZFS Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/361/feedback/Poojan%20-%20ZFS%20question.md) graceon - supermicro (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/361/feedback/graceon%20-%20supermicro.md) zenbum - groff (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/361/feedback/zenbum%20-%20groff.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** Special Guest: Warner Losh.
Thu, 30 Jul 2020 04:00:00 -0700
360: Full circle
Chasing a bad commit, New FreeBSD Core Team elected, Getting Started with NetBSD on the Pinebook Pro, FreeBSD on the Intel 10th Gen i3 NUC, pf table size check and change, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines Chasing a bad commit (https://vishaltelangre.com/chasing-a-bad-commit/) While working on a big project where multiple teams merge their feature branches frequently into a release Git branch, developers often run into situations where they find that some of their work have been either removed, modified or affected by someone else's work accidentally. It can happen in smaller teams as well. Two features could have been working perfectly fine until they got merged together and broke something. That's a highly possible case. There are many other cases which could cause such hard to understand and subtle bugs which even continuous integration (CI) systems running the entire test suite of our projects couldn't catch. We are not going to discuss how such subtle bugs can get into our release branch because that's just a wild territory out there. Instead, we can definitely discuss about how to find a commit that deviated from an expected outcome of a certain feature. The deviation could be any behaviour of our code that we can measure distinctively — either good or bad in general. New FreeBSD Core Team Elected (https://www.freebsdnews.com/2020/07/14/new-freebsd-core-team-elected/) The FreeBSD Project is pleased to announce the completion of the 2020 Core Team election. Active committers to the project have elected your Eleventh FreeBSD Core Team.! Baptiste Daroussin (bapt) Ed Maste (emaste) George V. Neville-Neil (gnn) Hiroki Sato (hrs) Kyle Evans (kevans) Mark Johnston (markj) Scott Long (scottl) Sean Chittenden (seanc) Warner Losh (imp) *** News Roundup Getting Started with NetBSD on the Pinebook Pro (https://bentsukun.ch/posts/pinebook-pro-netbsd/) If you buy a Pinebook Pro now, it comes with Manjaro Linux on the internal eMMC storage. Let’s install NetBSD instead! The easiest way to get started is to buy a decent micro-SD card (what sort of markings it should have is a science of its own, by the way) and install NetBSD on that. On a warm boot (i.e. when rebooting a running system), the micro-SD card has priority compared to the eMMC, so the system will boot from there. + A FreeBSD developer has borrowed some of the NetBSD code to get audio working on RockPro64 and Pinebook Pro: https://twitter.com/kernelnomicon/status/1282790609778905088 FreeBSD on the Intel 10th Gen i3 NUC (https://adventurist.me/posts/00300) I have ended up with some 10th Gen i3 NUC's (NUC10i3FNH to be specific) to put to work in my testbed. These are quite new devices, the build date on the boxes is 13APR2020. Before I figure out what their true role is (one of them might have to run linux) I need to install FreeBSD -CURRENT and see how performance and hardware support is. pf table size check and change (https://www.dragonflydigest.com/2020/06/29/24698.html) Did you know there’s a default size limit to pf’s state table? I did not, but it makes sense that there is one. If for some reason you bump into this limit (difficult for home use, I’d think), here’s how you change it (http://lists.dragonflybsd.org/pipermail/users/2020-June/381261.html) There is a table-entries limit specified, you can see current settings with 'pfctl -s all'. You can adjust the limits in the /etc/pf.conf file containing the rules with a line like this near the top: set limit table-entries 100000 + In the original mail thread, there is mention of the FreeBSD sysctl net.pf.request_maxcount, which controls the maximum number of entries that can be sent as a single ioctl(). This allows the user to adjust the memory limit for how big of a list the kernel is willing to allocate memory for. Beastie Bits tmux and bhyve (https://callfortesting.org/tmux/) Azure and FreeBSD (https://azuremarketplace.microsoft.com/en-us/marketplace/apps/thefreebsdfoundation.freebsd-12_1) Groff Tutorial (https://www.youtube.com/watch?v=bvkmnK6-qao&feature=youtu.be) *** ###Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Tarsnap Mastery (https://mwl.io/nonfiction/tools#tarsnap) Feedback/Questions Chris - ZFS Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/360/feedback/Chris%20-%20zfs%20question.md) Patrick - Tarsnap (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/360/feedback/Patrick%20-%20Tarsnap.md) Pin - pkgsrc (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/360/feedback/pin%20-%20pkgsrc.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 23 Jul 2020 05:00:00 -0700
359: Throwaway Browser
Throw-Away Browser on FreeBSD With "pot" within 5 minutes, OmniOS as OpenBSD guest with bhyve, BSD vs Linux distro development, My FreeBSD Laptop Build, FreeBSD CURRENT Binary Upgrades, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines Throw-Away Browser on FreeBSD With "pot" Within 5 Minutes (https://honeyguide.eu/posts/pot-throwaway-firefox/) pot is a great and relatively new jail management tool. It offers DevOps style provisioning and can even be used to provide Docker-like, scalable cloud services together with nomad and consul (more about this in Orchestrating jails with nomad and pot). OpenBSD guest with bhyve - OmniOS (https://www.pbdigital.org/omniosce/bhyve/openbsd/2020/06/08/bhyve-zones-omnios.html) Today I will be creating a OpenBSD guest via bhyve on OmniOS. I will also be adding a Pass Through Ethernet Controller so I can have a multi-homed guest that will serve as a firewall/router. This post will cover setting up bhyve on OmniOS, so it will also be a good introduction to bhyve. As well, I look into OpenBSD’s uEFI boot loader so if you have had trouble with this, then you are in the right place. News Roundup BSD versus Linux distribution development (https://distrowatch.com/weekly.php?issue=20200622#qa) Q: Comparing-apples-to-BSDs asks: I was reading one of the old articles from the archive. One of the things mentioned was how the BSDs have a distinct approach in terms of packaging the base system relative to userland apps, and that the Linux distros at the time were not following the same practice. Are there Linux distros that have adopted the same approach in modern times? If not, are there technical limitations that are preventing them from doing so, such as some distros supporting multiple kernel versions maybe? DistroWatch answers: In the article mentioned above, I made the observation that Linux distributions tend to take one of two approaches when it comes to packaging software. Generally a Linux distribution will either offer a rolling release, where virtually all packages are regularly upgraded to their latest stable releases, or a fixed release where almost all packages are kept at a set version number and only receive bug fixes for the life cycle of the distribution. Projects like Arch Linux and Void are popular examples of rolling, always-up-to-date distributions while Fedora and Ubuntu offer fixed platforms. My FreeBSD Laptop Build (https://corrupted.io/2020/06/21/my-freebsd-laptop-build.html) I have always liked Thinkpad hardware and when I started to do more commuting I decided I needed something that had a decent sized screen but fit well on a bus. Luckily about this time Lenovo gave me a nice gift in the Thinkpad X390. Its basically the famous X2xx series but with a 13” screen and smaller bezel. So with this laptop I figured it was time to actually put the docs together on how I got my FreeBSD workstation working on it. I will here in the near future have another post that will cover this for HardenedBSD as well since the steps are similar but have a few extra gotchas due to the extra hardening. FreeBSD CURRENT Binary Upgrades (http://up.bsd.lv) Disclaimer This proof-of-concept is not a publication of FreeBSD. Description up.bsd.lv is a proof-of-concept of binary updates for FreeBSD/amd64 CURRENT/HEAD to facilitate the exhaustive testing of FreeBSD and the bhyve hypervisor and OpenZFS 2.0 specifically. Updates are based on the SVN revisions of official FreeBSD Release Engineering bi-monthly snapshots. Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Karl - pfsense (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/359/Feedback/Karl%20-%20pfsense.md) Val - esxi question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/359/Feedback/Val%20-%20esxi%20question.md) lars - openbsd router hardware (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/359/Feedback/lars%20-%20openbsd%20router%20hardware.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 16 Jul 2020 04:00:00 -0700
358: OpenBSD Kubernetes Clusters
Yubikey-agent on FreeBSD, Managing Kubernetes clusters from OpenBSD, History of FreeBSD part 1, Running Jitsi-Meet in a FreeBSD Jail, Command Line Bug Hunting in FreeBSD, Game of Github, Wireguard official merged into OpenBSD, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines yubikey-agent on FreeBSD (https://kernelnomicon.org/?p=855) Some time ago Filippo Valsorda wrote yubikey-agent, seamless SSH agent for YubiKeys. I really like YubiKeys and worked on the FreeBSD support for U2F in Chromium and pyu2f, getting yubikey-agent ported looked like an interesting project. It took some hacking to make it work but overall it wasn’t hard. Following is the roadmap on how to get it set up on FreeBSD. The actual details depend on your system (as you will see) Manage Kubernetes clusters from OpenBSD (https://e1e0.net/manage-k8s-from-openbsd.html) This should work with OpenBSD 6.7. I write this while the source tree is locked for release, so even if I use -current this is as close as -current gets to -release Update 2020-06-05: we now have a port for kubectl. So, at least in -current things get a bit easier. News Roundup History of FreeBSD Part 1: Unix and BSD (https://klarasystems.com/articles/history-of-freebsd-unix-and-bsd/?utm_source=bsdnow) FreeBSD, a free and open-source Unix-like operating system has been around since 1993. However, its origins are directly linked to that of BSD, and further back, those of Unix. During this History of FreeBSD series, we will talk about how Unix came to be, and how Berkeley’s Unix developed at Bell Labs. Running Jitsi-Meet in a FreeBSD Jail (https://honeyguide.eu/posts/jitsi-freebsd/) Due to the situation with COVID-19 that also lead to people being confined to their homes in South Africa as well, we decided to provide a (freely usable of course) Jitsi Meet instance to the community being hosted in South Africa on our FreeBSD environment. That way, communities in South Africa and beyond have a free alternative to the commercial conferencing solutions with sometimes dubious security and privacy histories and at the same time improved user experience due to the lower latency of local hosting. + Grafana for Jitsi-Meet (https://honeyguide.eu/posts/jitsi-grafana/) Command Line Bug Hunting in FreeBSD (https://adventurist.me/posts/00301) FreeBSD uses bugzilla for tracking bugs, taking feature requests, regressions and issues in the Operating System. The web interface for bugzilla is okay, but if you want to do a lot of batch operations it is slow to deal with. We are planning to run a bugsquash on July 11th and that really needs some tooling to help any hackers that show up process the giant bug list we have. Beastie Bits Game of Github (https://glebbahmutov.com/game-of-github/) + Wireguard official merged into OpenBSD (https://marc.info/?l=openbsd-cvs&m=159274150512676&w=2) *** Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Florian : Lua for $HOME (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/358/feedback/Florian%20-%20Lua%20for%20%24HOME) Kevin : FreeBSD Source Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/358/feedback/Kevin%20-%20FreeBSD%20Source%20Question) Tom : HomeLabs (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/358/feedback/Tom%20-%20HomeLabs) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 09 Jul 2020 06:00:00 -0700
357: Study the Code
OpenBSD 6.7 on PC Engines, NetBSD code study, DRM Update on OpenBSD, Booting FreeBSD on HPE Microserver SATA port, 3 ways to multiboot, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines OpenBSD 6.7 on PC Engines APU4D4 (https://www.tumfatig.net/20200530/openbsd-6-7-on-pc-engines-apu4d4/) I just got myself a PC Engines APU4D4. I miss an OpenBSD box providing home services. It’s quite simple to install and run OpenBSD on this machine. And you can even update the BIOS from OpenBSD. NetBSD code study (http://silas.net.br/codereading/netbsd-code.html) News Roundup Booting FreeBSD off the HPE MicroServer Gen8 ODD SATA port (https://rubenerd.com/booting-freebsd-off-the-microserver-odd-sata-port/) My small homelab post generated a ton of questions and comments, most of them specific to running FreeBSD on the HP MicroServer. I’ll try and answer these over the coming week. Josh Paxton emailed to ask how I got FreeBSD booting on it, given the unconventional booting limitations of the hardware. I thought I wrote about it a few years ago, but maybe it’s on my proverbial draft heap. If you’re impatient, the script is in my lunchbox. 3 ways to multiboot (https://marc.info/?l=openbsd-misc&m=159146428705118&w=2) multiboot installation of a BSD system with other operating systems (OSs) on UEFI hardware is not officially supported by any of the popular Beastie Bits pfSense2.4.5-Release-p1 now available (https://www.netgate.com/blog/pfsense-2-4-5-release-p1-now-available.html) BSDCan 2020 TomSmyth - OpenBSD And OpenBGPD As ISP Controlplane (https://www.youtube.com/watch?v=_eOVlaYWqS8) OpenBSD DRM Update (https://undeadly.org/cgi?action=article;sid=20200608075708) *** ###Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions James - Apple T2 (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/357/feedback/James%20-%20Apple%20T2) Michael - Jordyns ZFS Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/357/feedback/Michael%20-%20Jordyns%20ZFS%20Question) Note from JT (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/357/feedback/Note%20from%20JT) Rob - FreeBSD Freindly Registrar (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/357/feedback/Rob%20-%20FreeBSD%20Freindly%20Registrar) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 02 Jul 2020 04:00:00 -0700
356: Dig in Deeper
TrueNAS is Multi-OS, Encrypted ZFS on NetBSD, FreeBSD’s new Code of Conduct, Gaming on OpenBSD, dig a little deeper, Hammer2 and periodic snapshots, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines TrueNAS is Multi-OS (https://www.ixsystems.com/blog/truenas-multi-os/) There was a time in history where all that mattered was an Operating System (OS) and the hardware it ran on — the “pre-software era”, if you will. Your hardware dictated the OS you used. Once software applications became prominent, your hardware’s OS determined the applications you could run. Application vendors were forced to juggle the burden of “portability” between OS platforms, choosing carefully the operating systems they’d develop their software to. Then, there were the great OS Wars of the 1990s, replete with the rampant competition, licensing battles, and nasty lawsuits, which more or less gave birth to the “open source OS” era. The advent of the hypervisor simultaneously gave way to the “virtual era” which set us on a path of agnosticism toward the OS. Instead of choosing from the applications available for your chosen OS, you could simply install another OS on the same hardware for your chosen application. The OS became nothing but a necessary cog in the stack. TrueNAS open storage enables this “post-OS era” with support for storage clients of all UNIX flavors, Linux, FreeBSD, Windows, MacOS, VMware, Citrix, and many others. Containerization has carried that mentality even further. An operating system, like the hardware that runs it, is now just thought of as part of the “infrastructure”. Encrypted ZFS on NetBSD 9.0, for a FreeBSD guy (https://rubenerd.com/encrypted-zfs-on-netbsd-9-for-a-freebsd-guy/) I had one of my other HP Microservers brought back from the office last week to help with this working-from-home world we’re in right now. I was going to wipe an old version of Debian Wheezy/Xen and install FreeBSD to mirror my other machines before thinking: why not NetBSD? News Roundup FreeBSD's New Code of Conduct (https://www.freebsd.org/internal/code-of-conduct.html) FreeBSD Announcement Email (https://raw.githubusercontent.com/BSDNow/bsdnow.tv/master/episodes/356/FBSD-CoC-Email) Gaming on OpenBSD (https://dataswamp.org/~solene/2020-06-05-openbsd-gaming.html) While no one would expect this, there are huge efforts from a small team to bring more games into OpenBSD. In fact, now some commercial games works natively now, thanks to Mono or Java. There are no wine or linux emulation layer in OpenBSD. Here is a small list of most well known games that run on OpenBSD: 'dig' a little deeper (https://vishaltelangre.com/dig-a-little-deeper/) I knew the existence of the dig command but didn't exactly know when and how to use it. Then, just recently I encountered an issue that allowed me to learn and make use of it. HAMMER2 and periodic snapshots (https://www.dragonflydigest.com/2020/06/15/24635.html) The first version of HAMMER took automatic snapshots, set within the config for each filesystem. HAMMER2 now also takes automatic snapshots, via periodic(8) like most every repeating task on your DragonFly system. + git: Implement periodic hammer2 snapshots (http://lists.dragonflybsd.org/pipermail/commits/2020-June/769247.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Cy - OpenSSL relicensing (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/356/feedback/Cy%20-%20OPenSSL%20relicensing.md) Christian - lagg vlans and iocage (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/356/feedback/Christian%20-%20lagg%20vlans%20and%20iocage) Brad - SMR (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/356/feedback/Brad%20-%20SMR) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 25 Jun 2020 04:00:00 -0700
355: Man Page Origins
Upgrading OpenBSD, Where do Unix man pages come from?, Help for NetBSD’s VAX port, FreeBSD on Dell Latitude 7390, PFS Tool changes in DragonflyBSD, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines How to Upgrade OpenBSD and Build a Kernel (https://cromwell-intl.com/open-source/openbsd-kernel.html) Let's see how to upgrade your OpenBSD system. Maybe you are doing this because the latest release just came out. If so, this is pretty simple: back up your data, boot from install media, and select "Upgrade" instead of "Install". But maybe the latest release has been out for a few months. Why would we go through the trouble of building and installing a new kernel or other core system components? Maybe some patches have been released to improve system security or stability. It is pretty easy to build and install a kernel on OpenBSD, easier and simpler in many ways than it is on Linux. The History of man pages (https://manpages.bsd.lv/history.html) Where do UNIX manpages come from? Who introduced the section-based layout of NAME, SYNOPSIS, and so on? And for manpage authors: where were those economical two- and three-letter instructions developed? VAX port needs help (http://blog.netbsd.org/tnf/entry/vax_port_needs_help) The VAX is the oldest machine architecture still supported by NetBSD. Unfortunately there is another challenge, totally outside of NetBSD, but affecting the VAX port big time: the compiler support for VAX is ... let's say sub-optimal. It is also risking to be dropped completely by gcc upstream. Now here is where people can help: there is a bounty campaign to finance a gcc hacker to fix the hardest and most immediate issue with gcc for VAX. Without this being resolved, gcc will drop support for VAX in a near future version. My new FreeBSD Laptop: Dell Latitude 7390 (http://www.daemonology.net/blog/2020-05-22-my-new-FreeBSD-laptop-Dell-7390.html) As a FreeBSD developer, I make a point of using FreeBSD whenever I can — including on the desktop. I've been running FreeBSD on laptops since 2004; this hasn't always been easy, but over the years I've found that the situation has generally been improving. One of the things we still lack is adequate documentation, however — so I'm writing this to provide an example for users and also Google bait in case anyone runs into some of the problems I had to address. PFS tool changes in DragonFly (https://www.dragonflydigest.com/2020/06/09/24612.html) HAMMER2 just became a little more DWIM: the pfs-list and pfs-delete directives will now look across all mounted filesystems, not just the current directory’s mount path. pfs-delete won’t delete any filesystem name that appears in more than one place, though + git: hammer2 - Enhance pfs-list and pfs-delete (http://lists.dragonflybsd.org/pipermail/commits/2020-June/769226.html) Enhance pfs-list to list PFSs available across all mounted hammer2 filesystems instead of just the current directory's mount. A specific mount may be specified via -s mountpt. Enhance pfs-delete to look for the PFS name across all mounted hammer2 filesystems instead of just the current directory's mount. As a safety, pfs-delete will refuse to delete PFS names which are duplicated across multiple mounts. A specific mount may be specified via -s mountpt. Beastie Bits BastilleBSD Templates (https://gitlab.com/bastillebsd-templates) Tianocore update (https://www.dragonflydigest.com/2020/06/08/24610.html) Reminder: FreeBSD Office Hours on June 24, 2020 (https://wiki.freebsd.org/OfficeHours) *** ###Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Niclas - Regarding the Lenovo E595 user from Episode 340 (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/355/feedback/Niclas%20-%20Regarding%20the%20Lenovo%20E595%20user%20from%20Episode%20340.md) Erik - What happened with the video (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/355/feedback/Erik%20-%20What%20happened%20with%20the%20video.md) Igor - Boot Environments (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/355/feedback/Igor%20-%20Boot%20Environments.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Thu, 18 Jun 2020 04:00:00 -0700
354: ZFS safekeeps data
FreeBSD 11.4-RC 2 available, OpenBSD 6.7 on a PineBook Pro 64, How OpenZFS Keeps Your Data Safe, Bringing FreeBSD to EC2, FreeBSD 2020 Community Survey, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines FreeBSD 11.4-RC2 Now Available (https://lists.freebsd.org/pipermail/freebsd-stable/2020-May/092320.html) The second RC build of the 11.4-RELEASE release cycle is now available. + 11.4-RELEASE notes (https://www.freebsd.org/releases/11.4R/relnotes.html) (still in progress at the time of recording) Install OpenBSD 6.7-current on a PineBook Pro 64 (https://xosc.org/pinebookpro.html) This document is work in progress and I'll update the date above once I change something. If you have something to add, remarks, etc please contact me. Preferably via Mastodon but other means of communication are also fine. News Roundup Understanding How OpenZFS Keeps Your Data Safe (https://www.ixsystems.com/blog/openzfs-keeps-your-data-safe/) Veteran technology writer Jim Salter wrote an excellent guide on the ZFS file system’s features and performance that we absolutely had to share. There’s plenty of information in the article for ZFS newbies and advanced users alike. Be sure to check out the article over at Ars Technica to learn more about ZFS concepts including pools, vdevs, datasets, snapshots, and replication, just to name a few. Bringing FreeBSD to ec2 (https://www.lastweekinaws.com/podcast/screaming-in-the-cloud/bringing-freebsd-to-ec2-with-colin-percival/) Colin is the founder of Tarsnap, a secure online backup service which combines the flexibility and scriptability of the standard UNIX "tar" utility with strong encryption, deduplication, and the reliability of Amazon S3 storage. Having started work on Tarsnap in 2006, Colin is among the first generation of users of Amazon Web Services, and has written dozens of articles about his experiences with AWS on his blog. FreeBSD 2020 Community Survey (https://www.research.net/r/freebsd-2020-community-survey) The FreeBSD Core Team invites you to complete the 2020 FreeBSD Community Survey. The purpose of this survey is to collect quantitative data from the public in order to help guide the project’s priorities and efforts. This is only the second time a survey has been conducted by the FreeBSD Project and your input is valued. The survey will remain open for 14 days and will close on June 16th at 17:00 UTC (Tuesday 10am PDT). Beastie Bits FreeBSD Project Proposals (https://www.freebsdfoundation.org/blog/submit-your-freebsd-project-proposal) TJ Hacking (https://www.youtube.com/channel/UCknj_nW8JWcFJOAbgd5_Zgw) Scotland Open Source podcast (https://twitter.com/ScotlandOSUM/status/1265987126321188864?s=19) Next FreeBSD Office Hours on June 24, 2020 (https://wiki.freebsd.org/OfficeHours) *** Feedback/Questions Tom - Writing for LPIrstudio (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/354/feedback/Tom%20-%20Wriitng%20for%20LPI.md) Luke - rstudio (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/354/feedback/Luke%20-%20rstudio.md) Matt - Vlans and Jails (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/354/feedback/Matt%20-%20Vlans%20and%20Jails.md) Morgan - Can I get some commentary on this issue (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/354/feedback/Morgan%20-%20Can%20I%20get%20some%20commentary%20on%20this%20issue.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 11 Jun 2020 04:00:00 -0700
353: ZFS on Ironwolf
Scheduling in NetBSD, ZFS vs. RAID on Ironwolf disks, OpenBSD on Microsoft Surface Go 2, FreeBSD for Linux sysadmins, FreeBSD on Lenovo T480, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/) Headlines Scheduling in NetBSD – Part 1 (https://manikishan.wordpress.com/2020/05/10/scheduling-in-netbsd-part-1/) In this blog, we will discuss about the 4.4BSD Thread scheduler one of the two schedulers in NetBSD and a few OS APIs that can be used to control the schedulers and get information while executing. ZFS versus RAID: Eight Ironwolf disks, two filesystems, one winner (https://arstechnica.com/gadgets/2020/05/zfs-versus-raid-eight-ironwolf-disks-two-filesystems-one-winner/) This has been a long while in the making—it's test results time. To truly understand the fundamentals of computer storage, it's important to explore the impact of various conventional RAID (Redundant Array of Inexpensive Disks) topologies on performance. It's also important to understand what ZFS is and how it works. But at some point, people (particularly computer enthusiasts on the Internet) want numbers. If you want to hear more from Jim, he has a new bi-weekly podcast with Allan and Joe Ressington over at 2.5admins.com (https://2.5admins.com/) News Roundup OpenBSD on the Microsoft Surface Go 2 (https://jcs.org/2020/05/15/surface_go2) I used OpenBSD on the original Surface Go back in 2018 and many things worked with the big exception of the internal Atheros WiFi. This meant I had to keep it tethered to a USB-C dock for Ethernet or use a small USB-A WiFi dongle plugged into a less-than-small USB-A-to-USB-C adapter. FreeBSD UNIX for Linux sysadmins (https://triosdevelopers.com/jason.eckert/blog/Entries/2020/5/2_FreeBSD_UNIX_for_Linux_sysadmins.html) If you’ve ever installed and explored another Linux distro (what Linux sysadmin hasn’t?!?), then exploring FreeBSD is going be somewhat similar with a few key differences. While there is no graphical installation, the installation process is straightforward and similar to installing a server-based Linux distro. Just make sure you choose the local_unbound package when prompted if you want to cache DNS lookups locally, as FreeBSD doesn’t have a built-in local DNS resolver that does this. Following installation, the directory structure is almost identical to Linux. Of course, you’ll notice some small differences here and there (e.g. regular user home directories are located under /usr/home instead of /home). Standard UNIX commands such as ls, chmod, find, which, ps, nice, ifconfig, netstat, sockstat (the ss command in Linux) are exactly as you’d expect, but with some different options here and there that you’ll see in the man pages. And yes, reboot and poweroff are there too. FreeBSD on the Lenovo Thinkpad T480 (https://www.davidschlachter.com/misc/t480-freebsd) Recently I replaced my 2014 MacBook Air with a Lenovo Thinkpad T480, on which I've installed FreeBSD, currently 12.1-RELEASE. This page documents my set-up along with various configuration tweaks and fixes. Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Benjamin - ZFS Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/353/feedback/Benjamin%20-%20ZFS%20Question.md) Brad - swappagergetswapspace errors (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/353/feedback/Brad%20-%20swap_pager_getswapspace%20errors.md) Brandon - gaming (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/353/feedback/Brandon%20-%20gaming.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 04 Jun 2020 08:00:00 -0700
352: Introducing Randomness
A brief introduction to randomness, logs grinding netatalk to a halt, NetBSD core team changes, Using qemu guest agent on OpenBSD kvm/qemu guests, WireGuard patchset for OpenBSD, FreeBSD 12.1 on a laptop, and more. Headlines Entropy (https://washbear.neocities.org/entropy.html) A brief introduction to randomness Problem: Computers are very predictable. This is by design. But what if we want them to act unpredictably? This is very useful if we want to secure our private communications with randomized keys, or not let people cheat at video games, or if we're doing statistical simulations or similar. Logs grinding Netatalk on FreeBSD to a hault (https://rubenerd.com/logs-grinding-netatalk-on-freebsd-to-a-hault/) I’ve heard it said the cobbler’s children walk barefoot. While posessing the qualities of a famed financial investment strategy, it speaks to how we generally put more effort into things for others than ourselves; at least in business. The HP Microserver I share with Clara is a modest affair compared to what we run at work. It has six spinning rust drives and two SSDs which are ZFS-mirrored; not even in a RAID 10 equivalent. This is underlaid with GELI for encryption, and served to our Macs with Netatalk over gigabit Ethernet with jumbo frames. News Roundup NetBSD Core Team Changes (https://mail-index.netbsd.org/netbsd-announce/2020/05/07/msg000314.html) Matt Thomas (matt@) has served on the NetBSD core team for over ten years, and has made many contributions, including ELF functionality, being the long-time VAX maintainer, gcc contributor, the generic pmap, and also networking functionality, and platform bring-up over the years. Matt has stepped down from the NetBSD core team, and we thank him for his many, extensive contributions. Robert Elz (kre@), a long time BSD contributor, has kindly accepted the offer to join the core team, and help us out with the benefit of his experience and advice over many years. Amongst other things, Robert has been maintaining our shell, liaising with the Austin Group, and bringing it up to date with modern functionality. Using qemu guest agent on OpenBSD kvm/qemu guests (https://undeadly.org/cgi?action=article;sid=20200514073852) In a post to the ports@ mailing list, Landry Breuil (landry@) shared some of his notes on using qemu guest agent on OpenBSD kvm/qemu guests. WireGuard patchset for OpenBSD (https://undeadly.org/cgi?action=article;sid=20200512080047) A while ago I wanted to learn more about OpenBSD development. So I picked a project, in this case WireGuard, to develop a native client for. Over the last two years, with many different iterations, and working closely with the WireGuard's creator (Jason [Jason A. Donenfeld - Ed.], CC'd), it started to become a serious project eventually reaching parity with other official implementations. Finally, we are here and I think it is time for any further development to happen inside the src tree. FreeBSD 12.1 on a laptop (https://dataswamp.org/~solene/2020-05-11-freebsd-workstation.html) I’m using FreeBSD again on a laptop for some reasons so expect to read more about FreeBSD here. This tutorial explain how to get a graphical desktop using FreeBSD 12.1. Beastie Bits List of useful FreeBSD Commands (https://medium.com/@tdebarbora/list-of-useful-freebsd-commands-92dffb8f8c57) Master Your Network With Unix Command Line Tools (https://itnext.io/master-your-network-with-unix-command-line-tools-790bdd3b3b87) Original Unix containers aka FreeBSD jails (https://twitter.com/nixcraft/status/1257674069387993088) Flashback : 2003 Article : Bill Joy's greatest gift to man – the vi editor (https://www.theregister.co.uk/2003/09/11/bill_joys_greatest_gift/) FreeBSD Journal March/April 2020 Filesystems: ZFS Encryption, FUSE, and more, plus Network Bridges (https://www.freebsdfoundation.org/past-issues/filesystems/) HAMBug meeting will be online again in June, so those from all over the world are welcome to join, June 9th (2nd Tuesday of each month) at 18:30 Eastern (https://www.hambug.ca/) Feedback/Questions + Lyubomir - GELI and ZFS (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/352/feedback/Lyubomir%20-%20GELI%20and%20ZFS.md) Patrick - powerd and powerd++ (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/352/feedback/Patrick%20-%20powerd%20and%20powerd%2B%2B.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 28 May 2020 05:00:00 -0700
351: Heaven: OpenBSD 6.7
Backup and Restore on NetBSD, OpenBSD 6.7 available, Building a WireGuard Jail with FreeBSD's standard tools, who gets to chown things and quotas, influence TrueNAS CORE roadmap, and more. Headlines Backup and Restore on NetBSD (https://e17i.github.io/articles-netbsd-backup/) Putting together the bits and pieces of a backup and restore concept, while not being rocket science, always seems to be a little bit ungrateful. Most Admin Handbooks handle this topic only within few pages. After replacing my old Mac Mini's OS by NetBSD, I tried to implement an automated backup, allowing me to handle it similarly to the time machine backups I've been using before. Suggestions on how to improve are always welcome. BSD Release: OpenBSD 6.7 (https://distrowatch.com/?newsid=10921) The OpenBSD project produces and operating system which places focus on portability, standardisation, code correctness, proactive security and integrated cryptography. The project's latest release is OpenBSD 6.7 which introduces several new improvements to the cron scheduling daemon, improvements to the web server daemon, and the top command now offers scrollable output. These and many more changes can be found in the project's release announcement: "This is a partial list of new features and systems included in OpenBSD 6.7. For a comprehensive list, see the changelog leading to 6.7. General improvements and bugfixes: Reduced the minimum allowed number of chunks in a CONCAT volume from 2 to 1, increasing the number of volumes which can be created on a single disk with bioctl(8) from 7 to 15. This can be used to create more partitions than previously. Rewrote the cron(8) flag-parsing code to be getopt-like, allowing tight formations like -ns and flag repetition. Renamed the 'options' field in crontab(5) to 'flags'. Added crontab(5) -s flag to the command field, indicating that only a single instance of the job should run concurrently. Added cron(8) support for random time values using the ~ operator. Allowed cwm(1) configuration of window size based on percentage of the master window during horizontal and vertical tiling actions." Release Announcement (https://marc.info/?l=openbsd-announce&m=158989783626149&w=2) Release Notes (https://www.openbsd.org/67.html) News Roundup Building a WireGuard Jail with the FreeBSD's Standard Tools (https://genneko.github.io/playing-with-bsd/networking/freebsd-wireguard-jail/) Recently, I had an opportunity to build a WireGuard jail on a FreeBSD 12.1 host. As it was really quick and easy to setup and it has been working completely fine for a month, I’d like to share my experience with anyone interested in this topic. The Unix divide over who gets to chown things, and (disk space) quotas (https://utcc.utoronto.ca/~cks/space/blog/unix/ChownDivideAndQuotas) One of the famous big splits between the BSD Unix world and the System V world is whether ordinary users can use chown (the command and the system call) to give away their own files. In System V derived Unixes you were generally allowed to; in BSD derived Unixes you weren't. Until I looked it up now to make sure, I thought that BSD changed this behavior from V7 and that V7 had an unrestricted chown. However, this turns out to be wrong; in V7 Unix, chown(2) was restricted to root only. You Can Influence the TrueNAS CORE Roadmap! (https://www.ixsystems.com/blog/truenas-bugs-and-suggestions/) As many of you know, we’ve historically had three ticket types available in our tracker: Bugs, Features, and Improvements, which are all fairly self-explanatory. After some discussion internally, we’ve decided to implement a new type of ticket, a “Suggestion”. These will be replacing Feature and Improvement requests for the TrueNAS Community, simplifying things down to two options: Bugs and Suggestions. This change also introduces a slightly different workflow than before. Beastie Bits FreeNAS Spare Parts Build: Testing ZFS With Imbalanced VDEVs and Mismatched Drives (https://www.youtube.com/watch?v=EFrlG3CUKFQ) TLSv1.3 server code enabled in LibreSSL in -current (https://undeadly.org/cgi?action=article;sid=20200512074150) Interview with Deb Goodkin (https://itsfoss.com/freebsd-interview-deb-goodkin/) *** Feedback/Questions Bostjan - WireGaurd (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/351/feedback/Bostjan%20-%20WireGaurd.md) Chad - ZFS Pool Design (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/351/feedback/Chad%20-%20ZFS%20Pool%20Design.md) Pedreo - Scale FreeBSD Jails (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/351/feedback/Pedreo%20-%20Scale%20FreeBSD%20Jails.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 21 May 2020 05:00:00 -0700
350: Speedy Bridges
5x if_bridge Performance Improvement, How Unix Won, Understanding VLAN Configuration on FreeBSD, Using bhyve PCI passthrough on OmniOS, TrueNAS 11.3-U2 Available, and more. Headlines 5x if_bridge Performance Improvement (https://www.freebsdfoundation.org/blog/500-if_bridge-performance-improvement/) With FreeBSD Foundation grant, Kristof Provost harnesses new parallel techniques to uncork performance bottleneck + Kristof also streamed some of his work, providing an interesting insight into how such development work happens + > https://www.twitch.tv/provostk/videos How Unix Won (https://blog.vivekhaldar.com/post/617189040564928512/how-unix-won) +> Unix has won in every conceivable way. And in true mythic style, it contains the seeds of its own eclipse. This is my subjective historical narrative of how that happened. I’m using the name “Unix” to include the entire family of operating systems descended from it, or that have been heavily influenced by it. That includes Linux, SunOS, Solaris, BSD, Mac OS X, and many, many others. Both major mobile OSs, Android and iOS, have Unix roots. Their billions of users dwarf those using clunky things like laptops and desktops, but even there, Windows is only the non-Unix viable OS. Almost everything running server-side in giant datacenters is Linux. How did Unix win? News Roundup Check logs of central syslog-ng log host on FreeBSD (https://blog.socruel.nu/freebsd/check-logs-of-syslog-ng-log-host-on-freebsd.html) This blog post continues where the blog post A central log host with syslog-ng on FreeBSD left off. Open source solutions to check syslog log messages exist, such as Logcheck or Logwatch. Although these are not to difficult to implement and maintain, I still found these to much. So I went for my own home grown solution to check the syslog messages of the SoCruel.NU central log host. And the solution presented in this blog post works pretty well for me! Understanding VLAN Configuration on FreeBSD (https://genneko.github.io/playing-with-bsd/networking/freebsd-vlan/) Until recently, I’ve never had a chance to use VLANs on FreeBSD hosts, though I sometimes configure them on ethernet switches. But when I was playing with vnet jails, I suddenly got interested in VLAN configuration on FreeBSD and experimented with it for some time. I wrote this short article to summarize my current understanding of how to configure VLANs on FreeBSD. Using bhyve PCI passthrough on OmniOS (https://www.cyber-tec.org/2019/05/29/using-bhyve-pci-passthrough-on-omnios/) Some hardware is not supported in illumos yet, but luckily there is bhyve which supports pci passthrough to any guest operating system. To continue with my OmniOS desktop on "modern" hardware I would love wifi support, so why not using a bhyve guest as router zone which provide the required drivers? TrueNAS 11.3-U2 is Generally Available (https://www.ixsystems.com/blog/truenas-11-3-u2-is-available/) TrueNAS 11.3-U2.1 is generally available as of 4/22/2020. This update is based on FreeNAS 11.3-U2 which has had over 50k deployments and received excellent community and third party reviews. The Release Notes are available on the iXsystems.com website. Beastie Bits HardenedBSD April 2020 Status Report (https://hardenedbsd.org/article/shawn-webb/2020-04-24/hardenedbsd-april-2020-status-report) NYC Bug’s Mailing List - Listing of open Dev Jobs (http://lists.nycbug.org/pipermail/jobs/2020-April/000553.html) Feedback/Questions Greg - Lenovo (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/350/feedback/Greg%20-%20Lenovos.md) Matt - BSD Packaging (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/350/feedback/Matt%20-%20BSD%20Packaging.md) Morgan - Performance (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/350/feedback/Morgan%20-%20Performance.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Thu, 14 May 2020 05:00:00 -0700
349: Entropy Overhaul
Encrypted Crash Dumps in FreeBSD, Time on Unix, Improve ZVOL sync write performance with a taskq, central log host with syslog-ng, NetBSD Entropy overhaul, Setting Up NetBSD Kernel Dev Environment, and more. Headlines EKCD - Encrypted Crash Dumps in FreeBSD (https://oshogbo.vexillium.org/blog/74/) Some time ago, I was describing how to configure networking crash dumps. In that post, I mentioned that there is also the possibility to encrypt crash dumps. Today we will look into this functionality. Initially, it was implemented during Google Summer of Code 2013 by my friend Konrad Witaszczyk, who made it available in FreeBSD 12. If you can understand Polish, you can also look into his presentation on BSD-PL on which he gave a comprehensive review of all kernel crash dumps features. The main issue with crash dumps is that they may include sensitive information available in memory during a crash. They will contain all the data from the kernel and the userland, like passwords, private keys, etc. While dumping them, they are written to unencrypted storage, so if somebody took out the hard drive, they could access sensitive data. If you are sending a crash dump through the network, it may be captured by third parties. Locally the data are written directly to a dump device, skipping the GEOM subsystem. The purpose of that is to allow a kernel to write a crash dump even in case a panic occurs in the GEOM subsystem. It means that a crash dump cannot be automatically encrypted with GELI. Time on Unix (https://venam.nixers.net/blog/unix/2020/05/02/time-on-unix.html) Time, a word that is entangled in everything in our lives, something we’re intimately familiar with. Keeping track of it is important for many activities we do. Over millennia we’ve developed different ways to calculate it. Most prominently, we’ve relied on the position the sun appears to be at in the sky, what is called apparent solar time. We’ve decided to split it as seasons pass, counting one full cycle of the 4 seasons as a year, a full rotation around the sun. We’ve also divided the passing of light to the lack thereof as days, a rotation of the earth on itself. Moving on to more precise clock divisions such as seconds, minutes, and hours, units that meant different things at different points in history. Ultimately, as travel got faster, the different ways of counting time that evolved in multiple places had to converge. People had to agree on what it all meant. See the article for more News Roundup Improve ZVOL sync write performance by using a taskq (https://github.com/openzfs/zfs/commit/0929c4de398606f8305057ca540cf577e6771c30) A central log host with syslog-ng on FreeBSD - Part 1 (https://blog.socruel.nu/freebsd/a-central-log-host-with-syslog-ng-on-freebsd.html) syslog-ng is the Swiss army knife of log management. You can collect logs from any source, process them in real time and deliver them to wide range of destinations. It allows you to flexibly collect, parse, classify, rewrite and correlate logs from across your infrastructure. This is why syslog-ng is the perfect solution for the central log host of my (mainly) FreeBSD based infrastructure. HEADS UP: NetBSD Entropy Overhaul (https://mail-index.netbsd.org/current-users/2020/05/01/msg038495.html) This week I committed an overhaul of the kernel entropy system. Please let me know if you observe any snags! For the technical background, see the thread on tech-kern a few months ago: https://mail-index.NetBSD.org/tech-kern/2019/12/21/msg025876.html. Setting Up NetBSD Kernel Dev Environment (https://adityapadala.com/2020/04/20/Setting-Up-NetBSD-Kernel-Dev-Environment/) I used T_PAGEFLT’s blog post as a reference for setting my NetBSD kernel development environment since his website is down I’m putting down the steps here so it would be helpful for starters. Beastie Bits You can now use ccache to speed up dsynth even more. (https://www.dragonflydigest.com/2020/05/04/24480.html) Improving libossaudio, and the future of OSS in NetBSD (http://blog.netbsd.org/tnf/entry/improving_libossaudio_and_the_future) DragonFlyBSD DHCPCD Import dhcpcd-9.0.2 with the following changes (http://lists.dragonflybsd.org/pipermail/commits/2020-April/769021.html) Reminder: watch this space for upcoming FreeBSD Office Hours, next is May 13th at 2pm Eastern, 18:00 UTC (https://wiki.freebsd.org/OfficeHours) Feedback/Questions Ghislain - ZFS Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/349/feedback/Ghislain%20-%20ZFS%20Question.md) Jake - Paypal Donations (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/349/feedback/Jake%20-%20Paypal%20Donations.md) Oswin - Hammer tutorial (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/349/feedback/Oswin%20-%20Hammer%20tutorial.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 07 May 2020 05:00:00 -0700
348: BSD Community Collections
FuryBSD 2020Q2 Images Available, Technical reasons to choose FreeBSD over GNU/Linux, Ars technica reviews GhostBSD, “TLS Mastery” sponsorships open, BSD community show their various collections, a tale of OpenBSD secure memory allocator internals, learn to stop worrying and love SSDs, and more. Headlines FuryBSD 2020Q2 Images Available for XFCE and KDE (https://www.furybsd.org/furybsd-2020-q2-images-are-available-for-xfce-and-kde/) The Q2 2020 images are not a visible leap forward but a functional leap forward. Most effort was spent creating a better out of box experience for automatic Ethernet configuration, working WiFi, webcam, and improved hypervisor support. Technical reasons to choose FreeBSD over GNU/Linux (https://unixsheikh.com/articles/technical-reasons-to-choose-freebsd-over-linux.html) Since I wrote my article "Why you should migrate everything from Linux to BSD" I have been wanting to write something about the technical reasons to choose FreeBSD over GNU/Linux and while I cannot possibly cover every single reason, I can write about some of the things that I consider worth noting. News Roundup + Not actually Linux distro review deux: GhostBSD (https://arstechnica.com/gadgets/2020/04/not-actually-linux-distro-review-deux-ghostbsd/) When I began work on the FreeBSD 12.1-RELEASE review last week, it didn't take long to figure out that the desktop portion wasn't going very smoothly. I think it's important for BSD-curious users to know of easier, gentler alternatives, so I did a little looking around and settled on GhostBSD for a follow-up review. GhostBSD is based on TrueOS, which itself derives from FreeBSD Stable. It was originally a Canadian distro, but—like most successful distributions—it has transcended its country of origin and can now be considered worldwide. Significant GhostBSD development takes place now in Canada, Italy, Germany, and the United States. “TLS Mastery” sponsorships open (https://mwl.io/archives/6265) My next book will be TLS Mastery, all about Transport Layer Encryption, Let’s Encrypt, OCSP, and so on. This should be a shorter book, more like my DNSSEC or Tarsnap titles, or the first edition of Sudo Mastery. I would like a break from writing doorstops like the SNMP and jails books. JT (our producer) shared his Open Source Retail Box Collection on twitter this past weekend and there was a nice response from a few in the BSD Community showing their collections: JT's post: https://twitter.com/q5sys/status/1251194823589138432 High Resolution Image to see the bottom shelf better: https://photos.smugmug.com/photos/i-9QTs2RR/0/f1742096/O/i-9QTs2RR.jpg Closeup of the BSD Section: https://twitter.com/q5sys/status/1251294290782928897 Others jumped in with their collections: Deb Goodkin's collection: https://twitter.com/dgoodkin/status/1251294016139743232 & https://twitter.com/dgoodkin/status/1251298125672660992 FreeBSD Frau's FreeBSD Collection: https://twitter.com/freebsdfrau/status/1251290430475350018 Jason Tubnor's OpenBSD Collection: https://twitter.com/Tubsta/status/1251265902214918144 Do you have a nice collection, take a picture and send it in! Tale of OpenBSD secure memory allocator internals - malloc(3) (https://bsdb0y.github.io/blog/deep-dive-into-the-OpenBSD-malloc-and-friends-internals-part-1.html) Hi there, It's been a very long time I haven't written anything after my last OpenBSD blogs, that is, OpenBSD Kernel Internals — Creation of process from user-space to kernel space. OpenBSD: Introduction to execpromises in the pledge(2) pledge(2): OpenBSD's defensive approach to OS Security So, again I started reading OpenBSD source codes with debugger after reducing my sleep timings and managing to get some time after professional life. This time I have picked one of my favourite item from my wishlist to learn and share, that is, OpenBSD malloc(3), secure allocator How I learned to stop worrying and love SSDs (https://www.ixsystems.com/community/threads/how-i-learned-to-stop-worrying-and-love-ssds.82617/) my home FreeNAS runs two pools for data. One RAIDZ2 with four spinning disk drives and one mirror with two SSDs. Toying with InfluxDB and Grafana in the last couple of days I found that I seem to have a constant write load of 1 Megabyte (!) per second on the SSDs. What the ...? So I run three VMs on the SSDs in total. One with Windows 10, two with Ubuntu running Confluence, A wiki essentially, with files for attachments and MySQL as the backend database. Clearly the writes had to stop when the wikis were not used at all, just sitting idle, right? Well even with a full query log and quite some experience in the operation of web applications I could not figure out what Confluence is doing (productively, no doubt) but trust me, it writes a couple of hundred kbytes to the database each second just sitting idle. My infrastructure as of 2019 (https://chown.me/blog/infrastructure-2019.html) I've wanted to write about my infrastructure for a while, but I kept thinking, "I'll wait until after I've done $nextthingonmytodo." Of course this cycle never ends, so I decided to write about its state at the end of 2019. Maybe I'll write an update on it in a couple of moons; who knows? For something different than our usual Beastie Bits… we bring you… We're all quarantined so lets install BSD on things! Install BSD on something this week, write it up and let us know about it, and maybe we'll feature you! Installation of NetBSD on a Mac Mini (https://e17i.github.io/articles-netbsd-install/) OpenBSD on the HP Envy 13 (https://icyphox.sh/blog/openbsd-hp-envy/) Install NetBSD on a Vintage Computer (https://www.rs-online.com/designspark/install-netbsd-on-a-vintage-computer) BSDCan Home Lab Panel recording session: May 5th at 18:00 UTC (https://twitter.com/allanjude/status/1251895348836143104) Allan started a series of FreeBSD Office Hours (https://wiki.freebsd.org/OfficeHours) BSDNow is going Independent After being part of Jupiter Broadcasting since we started back in 2013, BSDNow is moving to become independent. We extend a very large thank you to Jupiter Broadcasting and Linux Academy for hosting us for so many years, and allowing us to bring you over 100 episodes without advertisements. What does this mean for you, the listener? Not much will change, just make sure your subscription is via the RSS feed at BSDNow.tv rather than one of the Jupiter Broadcasting feeds. We will update you with more news as things settle out. Feedback/Questions Todd - LinusTechTips Claims about ZFS (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/348/feedback/Todd%20-%20LinusTechTips'%20claims%20on%20ZFS.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 30 Apr 2020 05:00:00 -0700
347: New Directions
Rethinking OpenBSD security, FreeBSD 2020 Q1 status report, the notion of progress and user interfaces, Comments about Thomas E. Dickey on NetBSD curses, making Unix a little more Plan9-like, Not-actually Linux distro review: FreeBSD, and more. Headlines Rethinking OpenBSD Security (https://flak.tedunangst.com/post/rethinking-openbsd-security) OpenBSD aims to be a secure operating system. In the past few months there were quite a few security errata, however. That’s not too unusual, but some of the recent ones were a bit special. One might even say bad. The OpenBSD approach to security has a few aspects, two of which might be avoiding errors and minimizing the risk of mistakes. Other people have other ideas about how to build secure systems. I think it’s worth examining whether the OpenBSD approach works, or if this is evidence that it’s doomed to failure. I picked a few errata, not all of them, that were interesting and happened to suit my narrative. FreeBSD 2020 Q1 Quarterly report (https://www.freebsd.org/news/status/report-2020-01-2020-03.html) Welcome, to the quarterly reports, of the future! Well, at least the first quarterly report from 2020. The new timeline, mentioned in the last few reports, still holds, which brings us to this report, which covers the period of January 2020 - March 2020. News Roundup The Notion of Progress and User Interfaces (https://herebeseaswines.net/essays/2020-04-13-the-notion-of-progress-and-user-interfaces) One trait of modern Western culture is the notion of progress. A view claiming, at large, everything is getting better and better. How should we think about progress? Both in general and regarding technology? Thomas E. Dickey on NetBSD curses (https://implementality.blogspot.com/2020/04/thomas-e-dickey-on-netbsd-curses.html) I was recently pointed at a web page on Thomas E. Dickeys site talking about NetBSD curses. It seems initially that the page was intended to be a pointer to some differences between ncurses and NetBSD curses and does appear to start off in this vein but it seems that the author has lost the plot as the document evolved and the tail end of it seems to be devolving into some sort of slanging match. I don't want to go through Mr. Dickey's document point by point, that would be tedious but I would like to pick out some of the things that I believe to be the most egregious. Please note that even though I am a NetBSD developer, the opinions below are my own and not the NetBSD projects. Making Unix a little more Plan9-like (https://woozle.org/papers/plan9.html) I’m not really interested in defending anything. I tried out plan9port and liked it, but I have to live in Unix land. Here’s how I set that up. A Warning The suckless community, and some of the plan9 communities, are dominated by jackasses. I hope that’s strong enough wording to impress the severity. Don’t go into IRC for help. Stay off the suckless email list. The software is great, the people who write it are well-spoken and well-reasoned, but for some reason the fandom is horrible to everyone. Not-actually Linux distro review: FreeBSD 12.1-RELEASE (https://arstechnica.com/gadgets/2020/04/not-actually-linux-distro-review-freebsd-12-1-release/) This month's Linux distro review isn't of a Linux distribution at all—instead, we're taking a look at FreeBSD, the original gangster of free Unix-like operating systems. The first FreeBSD release was in 1993, but the operating system's roots go further back—considerably further back. FreeBSD started out in 1992 as a patch-release of Bill and Lynne Jolitz's 386BSD—but 386BSD itself came from the original Berkeley Software Distribution (BSD). BSD itself goes back to 1977—for reference, Linus Torvalds was only seven years old then. Before we get started, I'd like to acknowledge something up front—our distro reviews include the desktop experience, and that is very much not FreeBSD's strength. FreeBSD is far, far better suited to running as a headless server than as a desktop! We're going to get a full desktop running on it anyway, because according to Lee Hutchinson, I hate myself—and also because we can't imagine readers wouldn't care about it. FreeBSD does not provide a good desktop experience, to say the least. But if you're hankering for a BSD-based desktop, don't worry—we're already planning a followup review of GhostBSD, a desktop-focused BSD distribution. Beastie Bits Wifi renewal restarted (https://blog.netbsd.org/tnf/entry/wifi_renewal_restarted) HAMMER2 and a quick start for DragonFly (https://www.dragonflydigest.com/2020/04/21/24421.html) Engineering NetBSD 9.0 (http://netbsd.org/~kamil/AsiaBSDCon/Kamil_Rytarowski_Engineering_NetBSD_9.0.pdf) Antivirus Protection using OPNsense Plugins (https://www.youtube.com/watch?v=94vz_-5lAkE) BSDCan Home Lab Panel recording session: May 5th at 18:00 UTC (https://twitter.com/allanjude/status/1251895348836143104) BSDNow is going Independent After being part of Jupiter Broadcasting since we started back in 2013, BSDNow is moving to become independent. We extend a very large thank you to Jupiter Broadcasting and Linux Academy for hosting us for so many years, and allowing us to bring you over 100 episodes without advertisements. LinuxAcademy is now under new leadership, and we understand that cutbacks needed to be made, and that BSD is not their core product. That does not mean your favourite BSD podcast is going away, we will continue and we expect things will not look much different. What does this mean for you, the listener? Not much will change, just make sure your subscription is via the RSS feed at BSDNow.tv rather than one of the Jupiter Broadcasting feeds. We will update you with more news as things settle out. Feedback/Questions Jordyn - ZFS Pool Problem (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/347/feedback/Jordyn%20zfs%20pool%20problem.md) debug - https://github.com/BSDNow/bsdnow.tv/raw/master/episodes/347/feedback/dbg.txt Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 23 Apr 2020 05:00:00 -0700
346: Core File Tales
Tales from a core file, Lenovo X260 BIOS Update with OpenBSD, the problem of Unix iowait and multi-CPU machines, Hugo workflow using FreeBSD Jails, Caddy, Restic; extending NetBSD-7 branch support, a tale of two hypervisor bugs, and more. Headlines Tales From a Core File - Lessons from the Unix stdio ABI: 40 Years Later (https://fingolfin.org/blog/20200327/stdio-abi.html) On the side, I’ve been wrapping up some improvements to the classic Unix stdio libraries in illumos. stdio contains the classic functions like fopen(), printf(), and the security nightmare gets(). While working on support for fmemopen() and friends I got to reacquaint myself with some of the joys of the stdio ABI and its history from 7th Edition Unix. With that in mind, let’s dive into this, history, and some mistakes not to repeat. While this is written from the perspective of the C programming language, aspects of it apply to many other languages. Update Lenovo X260 BIOS with OpenBSD (https://www.tumfatig.net/20200331/update-lenovo-x260-bios-with-openbsd/) My X260 only runs OpenBSD and has no CD driver. But I still need to upgrade its BIOS from time to time. And this is possible using the ISO BIOS image. First off all, you need to download the “BIOS Update (Bootable CD)” from the Lenovo Support Website. News Roundup The problem of Unix iowait and multi-CPU machines (https://utcc.utoronto.ca/~cks/space/blog/unix/IowaitAndMultipleCPUs) Various Unixes have had a 'iowait' statistic for a long time now (although I can't find a source for where it originated; it's not in 4.x BSD, so it may have come through System V and sar). The traditional and standard definition of iowait is that it's the amount of time the system was idle but had at least one process waiting on disk IO. Rather than count this time as 'idle' (as you would if you had a three-way division of CPU time between user, system, and idle), some Unixes evolved to count this as a new category, 'iowait'. My Latest Self Hosted Hugo Workflow using FreeBSD Jails, Caddy, Restic and More (https://www.jaredwolff.com/my-latest-self-hosted-hugo-workflow/) After hosting with Netlify for a few years, I decided to head back to self hosting. Theres a few reasons for that but the main reasoning was that I had more control over how things worked. In this post, i’ll show you my workflow for deploying my Hugo generated site (www.jaredwolff.com). Instead of using what most people would go for, i’ll be doing all of this using a FreeBSD Jails based server. Plus i’ll show you some tricks i’ve learned over the years on bulk image resizing and more. Let’s get to it. Extending support for the NetBSD-7 branch (http://blog.netbsd.org/tnf/entry/extending_support_for_the_netbsd) Typically, some time after releasing a new NetBSD major version (such as NetBSD 9.0), we will announce the end-of-life of the N-2 branch, in this case NetBSD-7. We've decided to hold off on doing that to ensure our users don't feel rushed to perform a major version update on any remote machines, possibly needing to reach the machine if anything goes wrong. Security fixes will still be made to the NetBSD-7 branch. We hope you're all safe. Stay home. Tale of two hypervisor bugs - Escaping from FreeBSD bhyve (http://phrack.org/papers/escaping_from_freebsd_bhyve.html) VM escape has become a popular topic of discussion over the last few years. A good amount of research on this topic has been published for various hypervisors like VMware, QEMU, VirtualBox, Xen and Hyper-V. Bhyve is a hypervisor for FreeBSD supporting hardware-assisted virtualization. This paper details the exploitation of two bugs in bhyve - FreeBSD-SA-16:32.bhyve (VGA emulation heap overflow) and CVE-2018-17160 (Firmware Configuration device bss buffer overflow) and some generic techniques which could be used for exploiting other bhyve bugs. Further, the paper also discusses sandbox escapes using PCI device passthrough, and Control-Flow Integrity bypasses in HardenedBSD 12-CURRENT Beastie Bits GhostBSD 20.02 Overview (https://www.youtube.com/watch?v=kFG-772WGwg) FuryBSD 12.1 Overview (https://www.youtube.com/watch?v=5V8680uoXxw) > Joe Maloney got in touch to say that the issues in the video and other ones found have since been fixed. Now that's community feedback in action, and an example of a developer who does his best to help the community. A great guy indeed. OS108-9.0 amd64 MATE released (https://forums.os108.org/d/27-os108-9-0-amd64-mate-released) FreeBSD hacking: carp panics & test (https://www.twitch.tv/videos/584064729) Inaugural FreeBSD Office Hours (https://www.youtube.com/watch?v=6qBm5NM3zTQ) Feedback/Questions Shody - systemd question (http://dpaste.com/2SAQDJJ#wrap) Ben - GELI and GPT (http://dpaste.com/1S0DGT3#wrap) Stig - DIY NAS (http://dpaste.com/2NGNZG5#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 16 Apr 2020 05:00:00 -0700
345: Switchers to BSD
NetBSD 8.2 is available, NextCloud on OpenBSD, X11 screen locking, NetBSD and RISC OS running parallel, community feedback about switching to BSD, and more. Headlines NetBSD 8.2 is available! (http://blog.netbsd.org/tnf/entry/netbsd_8_2_is_available) The third release in the NetBSD-8 is now available. This release includes all the security fixes in NetBSD-8 up until this point, and other fixes deemed important for stability. Some highlights include: x86: fixed regression in booting old CPUs x86: Hyper-V Gen.2 VM framebuffer support httpd(8): fixed various security issues ixg(4): various fixes / improvements x86 efiboot: add tftp support, fix issues on machines with many memory segments, improve graphics mode logic to work on more machines. Various kernel memory info leaks fixes Update expat to 2.2.8 Fix ryzen USB issues and support xHCI version 3.10. Accept root device specification as NAME=label. Add multiboot 2 support to x86 bootloaders. Fix for CVE-2019-9506: 'Key Negotiation of Bluetooth' attack. nouveau: limit the supported devices and fix firmware loading. radeon: fix loading of the TAHITI VCE firmware. named(8): stop using obsolete dnssec-lookaside. NextCloud on OpenBSD (https://h3artbl33d.nl/2020-nextcloud.html) NextCloud and OpenBSD are complementary to one another. NextCloud is an awesome, secure and private alternative for proprietary platforms, whereas OpenBSD forms the most secure and solid foundation to serve it on. Setting it up in the best way isn’t hard, especially using this step by step tutorial. Preface Back when this tutorial was initially written, things were different. The OpenBSD port relied on PHP 5.6 and there were no package updates. But the port improved (hats off, Gonzalo!) and package updates were introduced to the -stable branch (hats off, Solene!). A rewrite of this tutorial was long overdue. Right now, it is written for 6.6 -stable and will be updated once 6.7 is released. If you have any questions or desire some help, feel free to reach out. News Roundup X11 screen locking: a secure and modular approach (http://leahneukirchen.org/blog/archive/2020/01/x11-screen-locking-a-secure-and-modular-approach.html) For years I’ve been using XScreenSaver as a default, but I recently learned about xsecurelock and re-evaluated my screen-saving requirements NetBSD and RISC OS running parallel (http://www.update.uu.se/~micken/ronetbsd.html) I have been experimenting with running two systems at the same time on the RK3399 SoC. It all begun when I figured out how to switch to the A72 cpu for RISC OS. When the switch was done, the A53 cpu just continued to execute code. OK I thought why not give it something to do! My first step was to run some small programs. It worked! + Thanks to Tom Jones for the pointer to this article Several weeks ago we covered a story about switching from Linux to BSD. Benedict and JT asked for community feedback as to their thoughts on the matter. Allan was out that week, so this will give him an opportunity to chime in with his thoughts as well. Jamie - Dumping Linux for BSD (http://dpaste.com/0CH1YXQ#wrap) Matt - BSD Packaging (http://dpaste.com/2N68YPJ#wrap) Brad - Linux vs BS (http://dpaste.com/2SF9V38#wrap) MJ - Linux vs BSD Feedback (http://dpaste.com/0Z2ZT4V#wrap) Ben - Feedback for JT (http://dpaste.com/0B3M85X) Henrik - Why you should migrate everything to BSD (http://dpaste.com/3F36EQE#wrap) Beastie Bits ssh-copy-id now included (https://www.dragonflydigest.com/2020/04/06/24367.html) OPNsense 20.1.3 released (https://opnsense.org/opnsense-20-1-3-released/) A Collection of prebuilt BSD Cloud Images (https://bsd-cloud-image.org/) Instant terminal sharing (https://tmate.io/) Feedback/Questions Ales - Manually verify signature files for pkg package (http://dpaste.com/1EBWTK5#wrap) Shody - Yubikey (http://dpaste.com/340PM9Q#wrap) Mike - Site for hashes from old disks (http://dpaste.com/13W9SF0) Answer: https://docs.google.com/spreadsheets/d/19FmLs0jXxLkxAr0zwgdrXQd1qhbwvNHH6NvolvXKWTM/edit?usp=sharing Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 09 Apr 2020 05:00:00 -0700
344: Grains of Salt
Shell text processing, data rebalancing on ZFS mirrors, Add Security Headers with OpenBSD relayd, ZFS filesystem hierarchy in ZFS pools, speeding up ZSH, How Unix pipes work, grow ZFS pools over time, the real reason ifconfig on Linux is deprecated, clear your terminal in style, and more. Headlines Text processing in the shell (https://blog.balthazar-rouberol.com/text-processing-in-the-shell) This article is part of a self-published book project by Balthazar Rouberol and Etienne Brodu, ex-roommates, friends and colleagues, aiming at empowering the up and coming generation of developers. We currently are hard at work on it! One of the things that makes the shell an invaluable tool is the amount of available text processing commands, and the ability to easily pipe them into each other to build complex text processing workflows. These commands can make it trivial to perform text and data analysis, convert data between different formats, filter lines, etc. When working with text data, the philosophy is to break any complex problem you have into a set of smaller ones, and to solve each of them with a specialized tool. Rebalancing data on ZFS mirrors (https://jrs-s.net/2020/03/10/rebalancing-data-on-zfs-mirrors/) One of the questions that comes up time and time again about ZFS is “how can I migrate my data to a pool on a few of my disks, then add the rest of the disks afterward?” If you just want to get the data moved and don’t care about balance, you can just copy the data over, then add the new disks and be done with it. But, it won’t be distributed evenly over the vdevs in your pool. Don’t fret, though, it’s actually pretty easy to rebalance mirrors. In the following example, we’ll assume you’ve got four disks in a RAID array on an old machine, and two disks available to copy the data to in the short term. News Roundup Using OpenBSD relayd to Add Security Headers (https://web.archive.org/web/20191109121500/https://goblackcat.com/posts/using-openbsd-relayd-to-add-security-headers/) I am a huge fan of OpenBSD’s built-in httpd server as it is simple, secure, and quite performant. With the modern push of the large search providers pushing secure websites, it is now important to add security headers to your website or risk having the search results for your website downgraded. Fortunately, it is very easy to do this when you combine httpd with relayd. While relayd is principally designed for layer 3 redirections and layer 7 relays, it just so happens that it makes a handy tool for adding the recommended security headers. My website automatically redirects users from http to https and this gets achieved using a simple redirection in /etc/httpd.conf So if you have a configuration similar to mine, then you will still want to have httpd listen on the egress interface on port 80. The key thing to change here is to have httpd listen on 127.0.0.1 on port 443. How we set up our ZFS filesystem hierarchy in our ZFS pools (https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSOurContainerFilesystems) Our long standing practice here, predating even the first generation of our ZFS fileservers, is that we have two main sorts of filesystems, home directories (homedir filesystems) and what we call 'work directory' (workdir) filesystems. Homedir filesystems are called /h/NNN (for some NNN) and workdir filesystems are called /w/NNN; the NNN is unique across all of the different sorts of filesystems. Users are encouraged to put as much stuff as possible in workdirs and can have as many of them as they want, which mattered a lot more in the days when we used Solaris DiskSuite and had fixed-sized filesystems. Speeding up ZSH (https://blog.jonlu.ca/posts/speeding-up-zsh) https://web.archive.org/web/20200315184849/https://blog.jonlu.ca/posts/speeding-up-zsh I was opening multiple shells for an unrelated project today and noticed how abysmal my shell load speed was. After the initial load it was relatively fast, but the actual shell start up was noticeably slow. I timed it with time and these were the results. In the future I hope to actually recompile zsh with additional profiling techniques and debug information - keeping an internal timer and having a flag output current time for each command in a tree fashion would make building heat maps really easy. How do Unix Pipes work (https://www.vegardstikbakke.com/how-do-pipes-work-sigpipe/) Pipes are cool! We saw how handy they are in a previous blog post. Let’s look at a typical way to use the pipe operator. We have some output, and we want to look at the first lines of the output. Let’s download The Brothers Karamazov by Fyodor Dostoevsky, a fairly long novel. What we do to enable us to grow our ZFS pools over time (https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSHowWeGrowPools) In my entry on why ZFS isn't good at growing and reshaping pools, I mentioned that we go to quite some lengths in our ZFS environment to be able to incrementally expand our pools. Today I want to put together all of the pieces of that in one place to discuss what those lengths are. Our big constraint is that not only do we need to add space to pools over time, but we have a fairly large number of pools and which pools will have space added to them is unpredictable. We need a solution to pool expansion that leaves us with as much flexibility as possible for as long as possible. This pretty much requires being able to expand pools in relatively small increments of space. Linux maintains bugs: The real reason ifconfig on Linux is deprecated (https://blog.farhan.codes/2018/06/25/linux-maintains-bugs-the-real-reason-ifconfig-on-linux-is-deprecated/) In my third installment of FreeBSD vs Linux, I will discuss underlying reasons for why Linux moved away from ifconfig(8) to ip(8). In the past, when people said, “Linux is a kernel, not an operating system”, I knew that was true but I always thought it was a rather pedantic criticism. Of course no one runs just the Linux kernel, you run a distribution of Linux. But after reviewing userland code, I understand the significant drawbacks to developing “just a kernel” in isolation from the rest of the system. Clear Your Terminal in Style (https://adammusciano.com/2020/03/04/2020-03-04-clear-your-terminal-in-style/) if you’re someone like me who habitually clears their terminal, sometimes you want a little excitement in your life. Here is a way to do just that. This post revolves around the idea of giving a command a percent chance of running. While the topic at hand is not serious, this simple technique has potential in your scripts. Feedback/Questions Guy - AMD GPU Help (http://dpaste.com/2NEPDHB) MLShroyer13 - VLANs and Jails (http://dpaste.com/31KBNP4#wrap) Master One - ZFS Suspend/resume (http://dpaste.com/0DKM8CF#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 02 Apr 2020 05:00:00 -0700
343: FreeBSD, Corona: Fight!
Fighting the Coronavirus with FreeBSD, Wireguard VPN Howto in OPNsense, NomadBSD 1.3.1 available, fresh GhostBSD 20.02, New FuryBSD XFCE and KDE images, pf-badhost 0.3 released, and more. Headlines Fighting the Coronavirus with FreeBSD (https://www.leidinger.net/blog/2020/03/19/fighting-the-coronavirus-with-freebsd-foldinghome/) Here is a quick HOWTO for those who want to provide some FreeBSD based compute resources to help finding vaccines. UPDATE 2020-03-22: 0mp@ made a port out of this, it is in “biology/linux-foldingathome”. Per default it will now pick up some SARS-CoV‑2 (COVID-19) related folding tasks. There are some more config options (e.g. how much of the system resources are used). Please refer to the official Folding@Home site for more information about that. Be also aware that there is a big rise in compute resources donated to Folding@Home, so the pool of available work units may be empty from time to time, but they are working on adding more work units. Be patient. How to configure the Wireguard VPN in OPNsense (https://homenetworkguy.com/how-to/configure-wireguard-opnsense/) WireGuard is a modern designed VPN that uses the latest cryptography for stronger security, is very lightweight, and is relatively easy to set up (mostly). I say ‘mostly’ because I found setting up WireGuard in OPNsense to be more difficult than I anticipated. The basic setup of the WireGuard VPN itself was as easy as the authors claim on their website, but I came across a few gotcha's. The gotcha's occur with functionality that is beyond the scope of the WireGuard protocol so I cannot fault them for that. My greatest struggle was configuring WireGuard to function similarly to my OpenVPN server. I want the ability to connect remotely to my home network from my iPhone or iPad, tunnel all traffic through the VPN, have access to certain devices and services on my network, and have the VPN devices use my home's Internet connection. WireGuard behaves more like a SSH server than a typical VPN server. With WireGuard, devices which have shared their cryptographic keys with each other are able to connect via an encrypted tunnel (like a SSH server configured to use keys instead of passwords). The devices that are connecting to one another are referred to as “peer” devices. When the peer device is an OPNsense router with WireGuard installed, for instance, it can be configured to allow access to various resources on your network. It becomes a tunnel into your network similar to OpenVPN (with the appropriate firewall rules enabled). I will refer to the WireGuard installation on OPNsense as the server rather than a “peer” to make it more clear which device I am configuring unless I am describing the user interface because that is the terminology used interchangeably by WireGuard. The documentation I found on WireGuard in OPNsense is straightforward and relatively easy to understand, but I had to wrestle with it for a little while to gain a better understanding on how it should be configured. I believe it was partially due to differing end goals – I was trying to achieve something a little different than the authors of other wiki/blog/forum posts. Piecing together various sources of information, I finally ended up with a configuration that met the goals stated above. News Roundup NomadBSD 1.3.1 (https://nomadbsd.org/index.html#1.3.1) NomadBSD 1.3.1 has recently been made available. NomadBSD is a lightweight and portable FreeBSD distribution, designed to run on live on a USB flash drive, allowing you to plug, test, and play on different hardware. They have also started a forum as of yesterday, where you can ask questions and mingle with the NomadBSD community. Notable changes in 1.3.1 are base system upgraded to FreeBSD 12.1-p2. automatic network interface setup improved, image size increased to over 4GB, Thunderbird, Zeroconf, and some more listed below. GhostBSD 20.02 (https://ghostbsd.org/20.02_release_announcement) Eric Turgeon, main developer of GhostBSD, has announced version 20.02 of the FreeBSD based operating system. Notable changes are ZFS partition into the custom partition editor installer, allowing you to install alongside with Windows, Linux, or macOS. Other changes are force upgrade all packages on system upgrade, improved update station, and powerd by default for laptop battery performance. New FuryBSD XFCE and KDE images (https://www.furybsd.org/new-furybsd-12-1-based-images-are-available-for-xfce-and-kde/) This new release is now based on FreeBSD 12.1 with the latest FreeBSD quarterly packages. This brings XFCE up to 4.14, and KDE up to 5.17. In addition to updates this new ISO mostly addresses community bugs, community enhancement requests, and community pull requests. Due to the overwhelming amount of reports with GitHub hosting all new releases are now being pushed to SourceForge only for the time being. Previous releases will still be kept for archive purposes. pf-badhost 0.3 Released (https://www.geoghegan.ca/pfbadhost.html) pf-badhost is a simple, easy to use badhost blocker that uses the power of the pf firewall to block many of the internet's biggest irritants. Annoyances such as SSH and SMTP bruteforcers are largely eliminated. Shodan scans and bots looking for webservers to abuse are stopped dead in their tracks. When used to filter outbound traffic, pf-badhost blocks many seedy, spooky malware containing and/or compromised webhosts. Beastie Bits DragonFly i915 drm update (https://www.dragonflydigest.com/2020/03/23/24324.html) CShell is punk rock (http://blog.snailtext.com/posts/cshell-is-punk-rock.html) The most surprising Unix programs (https://minnie.tuhs.org/pipermail/tuhs/2020-March/020664.html) Feedback/Questions Master One - Torn between OpenBSD and FreeBSD (http://dpaste.com/102HKF5#wrap) Brad - Follow up to Linus ZFS story (http://dpaste.com/1VXQA2Y#wrap) Filipe Carvalho - Call for Portuguese BSD User Groups (http://dpaste.com/2H7S8YP) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 26 Mar 2020 05:00:00 -0700
342: Layout the DVA
OpenBSD Full disk encryption with coreboot and tianocore, FreeBSD 12.0 EOL, ZFS DVA layout, OpenBSD’s Go situation, AD updates requires changes in TrueNAS and FreeNAS, full name of FreeBSD’s root account, and more. Headlines OpenBSD Full Disk Encryption with CoreBoot and Tianocore Payload (https://functionallyparanoid.com/2020/03/07/openbsd-full-disk-encryption-with-coreboot-and-tianocore-payload/) It has been a while since I have posted here so I wanted to share something that was surprisingly difficult for me to figure out. I have a Thinkpad T440p that I have flashed with Coreboot 4.11 with some special patches that allow the newer machine to work. When I got the laptop, the default BIOS was UEFI and I installed two operating systems. Windows 10 with bitlocker full disk encryption on the “normal” drive (I replaced the spinning 2.5″ disk with an SSD) Ubuntu 19.10 on the m.2 SATA drive that I installed using LUKS full disk encryption I purchased one of those carriers for the optical bay that allows you to install a third SSD and so I did that with the intent of putting OpenBSD on it. Since my other two operating systems were running full disk encryption, I wanted to do the same on OpenBSD. See article for rest of story FreeBSD 12.0 EOL (https://lists.freebsd.org/pipermail/freebsd-announce/2020-February/001930.html) Dear FreeBSD community, As of February 29, 2020, FreeBSD 12.0 will reach end-of-life and will no longer be supported by the FreeBSD Security Team. Users of FreeBSD 12.0 are strongly encouraged to upgrade to a newer release as soon as possible. 12.1 Active release (https://www.freebsd.org/releases/12.1R/announce.html) 12.2 Release Schedule (https://www.freebsd.org/releases/12.2R/schedule.html) News Roundup Some effects of the ZFS DVA format on data layout and growing ZFS pools (https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSDVAFormatAndGrowth) One piece of ZFS terminology is DVA and DVAs, which is short for Data Virtual Address. For ZFS, a DVA is the equivalent of a block number in other filesystems; it tells ZFS where to find whatever data we're talking about. The short summary of what fields DVAs have and what they mean is that DVAs tell us how to find blocks by giving us their vdev (by number) and their byte offset into that particular vdev (and then their size). A typical DVA might say that you find what it's talking about on vdev 0 at byte offset 0x53a40ed000. There are some consequences of this that I hadn't really thought about until the other day. Right away we can see why ZFS has a problem removing a vdev; the vdev's number is burned into every DVA that refers to data on it. If there's no vdev 0 in the pool, ZFS has no idea where to even start looking for data because all addressing is relative to the vdev. ZFS pool shrinking gets around this by adding a translation layer that says where to find the portions of vdev 0 that you care about after it's been removed. Warning! Active Directory Security Changes Require TrueNAS and FreeNAS Updates. (https://www.ixsystems.com/blog/active-directory-truenas-and-freenas/) Critical Information for Current FreeNAS and TrueNAS Users Microsoft is changing the security defaults for Active Directory to eliminate some security vulnerabilities in its protocols. Unfortunately, these new security defaults may disrupt existing FreeNAS/TrueNAS deployments once Windows systems are updated. The Windows updates may appear sometime in March 2020; no official date has been announced as of yet. FreeNAS and TrueNAS users that utilize Active Directory should update to version 11.3 (or 11.2-U8) to avoid potential disruption of their networks when updating to the latest versions of Windows software after March 1, 2020. Version 11.3 has been released and version 11.2-U8 will be available in early March. Full name of the FreeBSD Root Account (https://www.geeklan.co.uk/?p=2457) NetBSD now has a users(7) and groups(7) manual. Looking into what entries existed in the passwd and group files I wondered about root’s full name who we now know as Charlie Root in the BSDs.... OpenBSD Go Situation (https://utcc.utoronto.ca/~cks/space/blog/programming/GoOpenBSDSituation) Over in the fediverse, Pete Zaitcev had a reaction to my entry on OpenBSD versus Prometheus for us: I don't think the situation is usually that bad. Our situation with Prometheus is basically a worst case scenario for Go on OpenBSD, and most people will have much better results, especially if you stick to supported OpenBSD versions. If you stick to supported OpenBSD versions, upgrading your machines as older OpenBSD releases fall out of support (as the OpenBSD people want you to do), you should not have any problems with your own Go programs. The latest Go release will support the currently supported OpenBSD versions (as long as OpenBSD remains a supported platform for Go), and the Go 1.0 compatibility guarantee means that you can always rebuild your current Go programs with newer versions of Go. You might have problems with compiled binaries that you don't want to rebuild, but my understanding is that this is the case for OpenBSD in general; it doesn't guarantee a stable ABI even for C programs (cf). If you use OpenBSD, you have to be prepared to rebuild your code after OpenBSD upgrades regardless of what language it's written in. Beastie Bits Test your TOR (http://lists.nycbug.org/pipermail/talk/2020-February/018174.html) OPNsense 20.1.1 released (https://opnsense.org/opnsense-20-1-1-released/) pkg for FreeBSD 1.13 (https://svnweb.freebsd.org/ports?view=revision&revision=525794) Feedback/Questions Bostjan writes in about Wireguard (http://dpaste.com/3WKG09D#wrap) Charlie has a followup to wpa_supplicant as lower class citizen (http://dpaste.com/0DDN99Q#wrap) Lars writes about LibreSSL as a positive example (http://dpaste.com/1N12HFB#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 19 Mar 2020 05:00:00 -0700
341: U-NAS-ification
FreeBSD on Power, DragonflyBSD 5.8 is here, Unifying FreeNAS/TrueNAS, OpenBSD vs. Prometheus and Go, gcc 4.2.1 removed from FreeBSD base, and more. Headlines FreeBSD on Power (https://www.freebsdfoundation.org/blog/power-to-the-people-making-freebsd-a-first-class-citizen-on-power/) The power and promise of all open source software is freedom. Another way to express freedom is choice — choice of platforms, deployment models, stacks, configurations, etc. The FreeBSD Foundation is dedicated to supporting and promoting the FreeBSD Project and community worldwide. But, what does this mean, exactly, you may wonder. The truth is it means many different things, but in all cases the Foundation acts to expand freedom and choice so that FreeBSD users have the power to serve their varied compute needs. This blog tells the story of one specific way the Foundation helps a member of the community provide greater hardware choice for all FreeBSD users. Dragonfly 5.8 (https://www.dragonflybsd.org/release58/) DragonFly version 5.8 brings a new dsynth utility for building your own binary dports packages, plus significant support work to speed up that build - up to and including the entire collection. Additional progress has been made on GPU and signal support. The details of all commits between the 5.6 and 5.8 branches are available in the associated commit messages for 5.8.0rc1 and 5.8.0. Also see /usr/src/UPDATING for specific file changes in PAM. See article for rest of information 2nd HamBUG meeting recap (https://www.hambug.ca/) The second meeting of the Hamilton BSD Users Group took place last night The next meeting is scheduled for the 2nd Tuesday of the month, April 14th 2020 News Roundup FreeNAS/TrueNAS Brand Unification (https://www.ixsystems.com/blog/freenas-truenas-unification/) FreeNAS and TrueNAS have been separate-but-related members of the #1 Open Source storage software family since 2012. FreeNAS is the free Open Source version with an expert community and has led the pursuit of innovations like Plugins and VMs. TrueNAS is the enterprise version for organizations of all sizes that need additional uptime and performance, as well as the enterprise-grade support necessary for critical data and applications. From the beginning at iXsystems, we’ve developed, tested, documented, and released both as separate products, even though the vast majority of code is shared. This was a deliberate technical decision in the beginning but over time became less of a necessity and more of “just how we’ve always done it”. Furthermore, to change it was going to require a serious overhaul to how we build and package both products, among other things, so we continued to kick the can down the road. As we made systematic improvements to development and QA efficiency over the past few years, the redundant release process became almost impossible to ignore as our next major efficiency roadblock to overcome. So, we’ve finally rolled up our sleeves. With the recent 11.3 release, TrueNAS gained parity with FreeNAS on features like VMs and Plugins, further homogenizing the code. Today, we announce the next phase of evolution for FreeNAS and TrueNAS. OpenBSD versus Prometheus (and Go). (https://utcc.utoronto.ca/~cks/space/blog/sysadmin/OpenBSDVsPrometheusAndGo) We have a decent number of OpenBSD machines that do important things (and that have sometimes experienced problems like running out of disk space), and we have a Prometheus based metrics and monitoring system. The Prometheus host agent has enough support for OpenBSD to be able to report on critical metrics, including things like local disk space. Despite all of this, after some investigation I've determined that it's not really sensible to even try to deploy the host agent on our OpenBSD machines. This is due to a combination of factors that have at their root OpenBSD's lack of ABI stability FreeBSD removed gcc from base (https://svnweb.freebsd.org/base?view=revision&revision=358454) As described in Warner's email message[1] to the FreeBSD-arch mailing list we have reached GCC 4.2.1's retirement date. At this time all supported architectures either use in-tree Clang, or rely on external toolchain (i.e., a contemporary GCC version from ports). GCC 4.2.1 was released July 18, 2007 and was imported into FreeBSD later that year, in r171825. GCC has served us well, but version 4.2.1 is obsolete and not used by default on any architecture in FreeBSD. It does not support modern C and does not support arm64 or RISC-V. Beastie Bits New Archive location for Dragonfly 4.x (https://www.dragonflydigest.com/2020/03/10/24276.html) A dead simple git cheat sheet (https://hub.iwebthings.com/a-dead-simple-git-cheatsheet/) Xorg 1.20.7 on HardenedBSD Comes with IE/RELRO+BIND_NOW/CFI/SafeStack Protections (https://twitter.com/lattera/status/1233412881569415168) Feedback/Questions Niclas writes in Regarding the Lenovo E595 user (episode 340) (http://dpaste.com/2YJ6PFW#wrap) Lyubomir writes about GELI and ZFS (http://dpaste.com/1S0DGT3#wrap) Peter writes in about scaling FreeBSD jails (http://dpaste.com/2FSZQ8V#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 12 Mar 2020 05:00:00 -0700
340: Check My Sums
Why ZFS is doing filesystem checksumming right, better TMPFS throughput performance on DragonFlyBSD, reshaping pools with ZFS, PKGSRC on Manjaro aarch64 Pinebook-pro, central log host with syslog-ng on FreeBSD, and more. Headlines Checksumming in filesystems, and why ZFS is doing it right (https://oshogbo.vexillium.org/blog/73/) One of the best aspects of ZFS is its reliability. This can be accomplished using a few features like copy-on-write approach and checksumming. Today we will look at how ZFS does checksumming and why it does it the proper way. Most of the file systems don’t provide any integrity checking and fail in several scenarios: Data bit flips - when the data that we wanted to store are bit flipped by the hard drives, or cables, and the wrong data is stored on the hard drive. Misdirected writes - when the CPU/cable/hard drive will bit flip a block to which the data should be written. Misdirected read - when we miss reading the block when a bit flip occurred. Phantom writes - when the write operation never made it to the disk. For example, a disk or kernel may have some bug that it will return success even if the hard drive never made the write. This problem can also occur when data is kept only in the hard drive cache. Checksumming may help us detect errors in a few of those situations. DragonFlyBSD Improves Its TMPFS Implementation For Better Throughput Performance (https://www.phoronix.com/scan.php?page=news_item&px=DragonFlyBSD-TMPFS-Throughput) It's been a while since last having any new magical optimizations to talk about by DragonFlyBSD lead developer Matthew Dillon, but on Wednesday he landed some significant temporary file-system "TMPFS" optimizations for better throughput including with swap. Of several interesting commits merged tonight, the improved write clustering is a big one. In particular, "Reduces low-memory tmpfs paging I/O overheads by 4x and generally increases paging throughput to SSD-based swap by 2x-4x. Tmpfs is now able to issue a lot more 64KB I/Os when under memory pressure." https://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/4eb0bb82efc8ef32c4357cf812891c08d38d8860 There's also a new tunable in the VM space as well as part of his commits on Wednesday night. This follows a lot of recent work on dsynth, improved page-out daemon pipelining, and other routine work. https://gitweb.dragonflybsd.org/dragonfly.git/commit/bc47dbc18bf832e4badb41f2fd79159479a7d351 This work is building up towards the eventual DragonFlyBSD 5.8 while those wanting to try the latest improvements right away can find their daily snapshots. News Roundup Why ZFS is not good at growing and reshaping pools (or shrinking them) (https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWhyNoRealReshaping) recently read Mark McBride's Five Years of Btrfs (via), which has a significant discussion of why McBride chose Btrfs over ZFS that boils down to ZFS not being very good at evolving your pool structure. You might doubt this judgment from a Btrfs user, so let me say as both a fan of ZFS and a long term user of it that this is unfortunately quite true; ZFS is not a good choice if you want to modify your pool disk layout significantly over time. ZFS works best if the only change in your pools that you do is replacing drives with bigger drives. In our ZFS environment we go to quite some lengths to be able to expand pools incrementally over time, and while this works it both leaves us with unbalanced pools and means that we're basically forced to use mirroring instead of RAIDZ. (An unbalanced pool is one where some vdevs and disks have much more data than others. This is less of an issue for us now that we're using SSDs instead of HDs.) Using PKGSRC on Manjaro Linux aarch64 Pinebook-pro (https://astr0baby.wordpress.com/2020/02/09/using-pkgsrc-on-manjaro-linux-aarch64-pinebook-pro/) I wanted to see how pkgsrc works on aarch64 Linux Manjaro since it is a very mature framework that is very portable and supported by many architectures – pkgsrc (package source) is a package management system for Unix-like operating systems. It was forked from the FreeBSD ports collection in 1997 as the primary package management system for NetBSD. One might question why use pkgsrc on Arch based Manjaro, since the pacman package repository is very good on its own. I see alternative pkgsrc as a good automated build framework that offers a way to produce independent build environment /usr/pkg that does not interfere with the current Linux distribution in any way (all libraries are statically built) I have used the latest Manjaro for Pinebookpro and standard recommended tools as mentioned here https://wiki.netbsd.org/pkgsrc/howtousepkgsrcon_linux/ A Central Log Host with syslog-ng on FreeBSD Part 1 (https://blog.socruel.nu/freebsd/a-central-log-host-with-syslog-ng-on-freebsd.html) syslog-ng is the Swiss army knife of log management. You can collect logs from any source, process them in real time and deliver them to wide range of destinations. It allows you to flexibly collect, parse, classify, rewrite and correlate logs from across your infrastructure. This is why syslog-ng is the perfect solution for the central log host of my (mainly) FreeBSD based infrastructure. Part 2 (https://blog.socruel.nu/freebsd/check-logs-of-syslog-ng-log-host-on-freebsd.html) This blog post continues where the blog post A central log host with syslog-ng on FreeBSD left off. Open source solutions to check syslog log messages exist, such as Logcheck or Logwatch. Although these are not too difficult to implement and maintain, I still found these to much. So I went for my own home grown solution to check the syslog messages of the SoCruel.NU central log host. Beastie Bits FreeBSD at Linux Conf 2020 session videos now online (https://mirror.linux.org.au/pub/linux.conf.au/2020/room_9/Tuesday/) Unlock your laptop with your phone (https://vermaden.wordpress.com/2020/01/09/freebsd-desktop-part-20-configuration-unlock-your-laptop-with-phone/) Managing a database of vulnerabilities for a package system: the pkgsrc study (https://www.netbsd.org/gallery/presentations/leot/itasec20/pkgsrc-security.pdf) Hamilton BSD User group will meet again on March 10th](http://studybsd.com/) CharmBUG Meeting: March 24th 7pm in Severn, MD (https://www.meetup.com/en-AU/CharmBUG/events/268251508/) *** Feedback/Questions Andrew - ZFS feature Flags (http://dpaste.com/2YM23C0#wrap) Sam - TwinCat BSD (http://dpaste.com/0FCZV6R) Dacian - Freebsd + amdgpu + Lenovo E595 (http://dpaste.com/1R7F1JN#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 05 Mar 2020 05:00:00 -0800
339: BSD Fundraising
Meet FuryBSD, NetBSD 9.0 has been released, OpenBSD Foundation 2019 campaign wrapup, a retrospective on OmniOS ZFS-based NFS fileservers, NetBSD Fundraising 2020 goal, OpenSSH 8.2 released, and more.## Headlines Meet FuryBSD: A New Desktop BSD Distribution (https://itsfoss.com/furybsd/) At its heart, FuryBSD is a very simple beast. According to the site, “FuryBSD is a back to basics lightweight desktop distribution based on stock FreeBSD.” It is basically FreeBSD with a desktop environment pre-configured and several apps preinstalled. The goal is to quickly get a FreeBSD-based system running on your computer. You might be thinking that this sounds a lot like a couple of other BSDs that are available, such as NomadBSD and GhostBSD. The major difference between those BSDs and FuryBSD is that FuryBSD is much closer to stock FreeBSD. For example, FuryBSD uses the FreeBSD installer, while others have created their own installers and utilities. As it states on the site, “Although FuryBSD may resemble past graphical BSD projects like PC-BSD and TrueOS, FuryBSD is created by a different team and takes a different approach focusing on tight integration with FreeBSD. This keeps overhead low and maintains compatibility with upstream.” The lead dev also told me that “One key focus for FuryBSD is for it to be a small live media with a few assistive tools to test drivers for hardware.” Currently, you can go to the FuryBSD homepage and download either an XFCE or KDE LiveCD. A GNOME version is in the works. NetBSD 9.0 (https://www.netbsd.org/releases/formal-9/NetBSD-9.0.html) The NetBSD Project is pleased to announce NetBSD 9.0, the seventeenth major release of the NetBSD operating system. This release brings significant improvements in terms of hardware support, quality assurance, security, along with new features and hundreds of bug fixes. Here are some highlights of this new release. News Roundup OpenBSD Foundation 2019 campaign wrapup (http://undeadly.org/cgi?action=article;sid=20200217001107) Our target for 2019 was CDN$300K. Our community's continued generosity combined with our corporate donors exceeded that nicely. In addition we received the largest single donation in our history, CDN$380K from Smartisan. The return of Google was another welcome event. Altogether 2019 was our most successful campaign to date, yielding CDN$692K in total. We thank all our donors, Iridium (Smartisan), Platinum (Yandex, Google), Gold (Microsoft, Facebook) Silver (2Keys) and Bronze (genua, Thinkst Canary). But especially our community of smaller donors whose contributions are the bedrock of our support. Thank you all! OpenBSD Foundation 2019 Fundraising Goal Exceeded (https://www.openbsdfoundation.org/campaign2019.html) A retrospective on our OmniOS ZFS-based NFS fileservers (https://utcc.utoronto.ca/~cks/space/blog/solaris/OmniOSFileserverRetrospective) Our OmniOS fileservers have now been out of service for about six months, which makes it somewhat past time for a retrospective on them. Our OmniOS fileservers followed on our Solaris fileservers, which I wrote a two part retrospective on (part 1, part 2), and have now been replaced by our Linux fileservers. To be honest, I have been sitting on my hands about writing this retrospective because we have mixed feelings about our OmniOS fileservers. I will put the summary up front. OmniOS worked reasonably well for us over its lifespan here and looking back I think it was almost certainly the right choice for us at the time we made that choice (which was 2013 and 2014). However it was not without issues that marred our experience with it in practice, although not enough to make me regret that we ran it (and ran it for as long as we did). Part of our issues are likely due to a design mistake in making our fileservers too big, although this design mistake was probably magnified when we were unable to use Intel 10G-T networking in OmniOS. On the one hand, our OmniOS fileservers worked, almost always reliably. Like our Solaris fileservers before them, they ran quietly for years without needing much attention, delivering NFS fileservice to our Ubuntu servers; specifically, we ran them for about five years (2014 through 2019, although we started migrating away at the end of 2018). Over this time we had only minor hardware issues and not all that many disk failures, and we suffered no data loss (with ZFS checksums likely saving us several times, and certainly providing good reassurances). Our overall environment was easy to manage and was pretty much problem free in the face of things like failed disks. I'm pretty sure that our users saw a NFS environment that was solid, reliable, and performed well pretty much all of the time, which is the important thing. So OmniOS basically delivered the fileserver environment we wanted. NetBSD Fundraising 2020 goal (http://blog.netbsd.org/tnf/entry/fundraising_2020) Is it really more than 10 years since we last had an official fundraising drive? Looking at old TNF financial reports I noticed that we have been doing quite well financially over the last years, with a steady stream of small and medium donations, and most of the time only moderate expenditures. The last fundraising drive back in 2009 was a giant success, and we have lived off it until now. OpenSSH 8.2 released February 14, 2020 (http://www.openssh.com/txt/release-8.2) OpenSSH 8.2 was released on 2020-02-14. It is available from the mirrors listed at https://www.openssh.com/. OpenSSH is a 100% complete SSH protocol 2.0 implementation and includes sftp client and server support. Once again, we would like to thank the OpenSSH community for their continued support of the project, especially those who contributed code or patches, reported bugs, tested snapshots or donated to the project. More information on donations may be found at: https://www.openssh.com/donations.html Beastie Bits FreeNAS vs. Unraid: GRUDGE MATCH! (https://www.youtube.com/watch?v=aXsRIrC5bjg) Unix Toolbox (http://cb.vu/unixtoolbox.xhtml) Rigs of Rods - OpenBSD Physics Game (https://docs.rigsofrods.org/) NYCBug - Dr Vixie (http://dpaste.com/0V35MAB#wrap) Hamilton BSD User group will meet again on March 10th](http://studybsd.com/) BSD Stockholm - Meetup March 3rd 2020 (https://www.meetup.com/BSD-Users-Stockholm/events/267873938/) Feedback/Questions Shirkdog - Question (http://dpaste.com/36E2BZ1) Master One - ZFS + Suspend/resume (http://dpaste.com/3B9M814#wrap) Micah Roth - ZFS write caching (http://dpaste.com/0D4GDX1#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 27 Feb 2020 05:00:00 -0800
338: iocage in Jail
Distrowatch reviews FuryBSD, LLDB on i386 for NetBSD, wpa_supplicant as lower-class citizen, KDE on FreeBSD updates, Travel Grant for BSDCan open, ZFS dataset for testing iocage within a jail, and more. Headlines Distrowatch Fury BSD Review (https://distrowatch.com/weekly.php?issue=20200127#furybsd) FuryBSD is the most recent addition to the DistroWatch database and provides a live desktop operating system based on FreeBSD. FuryBSD is not entirely different in its goals from NomadBSD, which we discussed recently. I wanted to take this FreeBSD-based project for a test drive and see how it compares to NomadBSD and other desktop-oriented projects in the FreeBSD family. FuryBSD supplies hybrid ISO/USB images which can be used to run a live desktop. There are two desktop editions currently, both for 64-bit (x86_64) machines: Xfce and KDE Plasma. The Xfce edition is 1.4GB in size and is the flavour I downloaded. The KDE Plasma edition is about 3.0GB in size. My fresh install of FuryBSD booted to a graphical login screen. From there I could sign into my account, which brings up the Xfce desktop. The installed version of Xfce is the same as the live version, with a few minor changes. Most of the desktop icons have been removed with just the file manager launchers remaining. The Getting Started and System Information icons have been removed. Otherwise the experience is virtually identical to the live media. FuryBSD uses a theme that is mostly grey and white with creamy yellow folder icons. The application menu launchers tend to have neutral icons, neither particularly bright and detailed or minimal. LLDB now works on i386 (http://blog.netbsd.org/tnf/entry/lldb_now_works_on_i386) Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February 2019, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, fixing watchpoint and threading support. The original NetBSD port of LLDB was focused on amd64 only. In January, I have extended it to support i386 executables. This includes both 32-bit builds of LLDB (running natively on i386 kernel or via compat32) and debugging 32-bit programs from 64-bit LLDB. News Roundup wpa_supplicant is definitely a lower-class citizen, sorry (https://marc.info/?l=openbsd-misc&m=158068418807352&w=2) wpa_supplicant is definitely a lower-class citizen, sorry. I increasingly wonder why this stuff matters; transit costs are so much lower than the period when eduroam was setup, and their reliance on 802.11x is super weird in a world where, for the most part + entire cities have open wifi in their downtown core + edu vs edu+transit split horizon problems have to be solved anyways + many universities have parallel open wifi + rate limiting / fare-share approaches for the open-net, on unmetered + flat-rate solves the problem + LTE hotspot off a phone isn't a rip off anymore + other open networks exist essentially no one else feels compelled to do use 802.11x for a so called "semi-open access network", so I think they've lost the plot on friction vs benefit. (we've held hackathons at EDU campus that are locked down like that, and in every case we've said no way, gotten a wire with open net, and built our own wifi. we will not subject our developers to that extra complexity). KDE FreeBSD Updates Feb 2020 (https://euroquis.nl/freebsd/2020/02/08/freebsd.html) Some bits and bobs from the KDE FreeBSD team in february 2020. We met at the FreeBSD devsummit before FOSDEM, along with other FreeBSD people. Plans were made, schemes were forged, and Groff the Goat was introduced to some new people. The big ticket things: Frameworks are at 5.66 Plasma is at 5.17.5 (the beta 5.18 hasn’t been tried) KDE release service has landed 19.12.2 (same day it was released) Developer-centric: KDevelop is at 5.5.0 KUserfeedback landed its 1.0.0 release CMake is 3.16.3 Applications: Musescore is at 3.4.2 Elisa now part of the KDE release service updates Fuure work: KIO-Fuse probably needs extra real-world testing on FreeBSD. I don’t have that kind of mounts (just NFS in /etc/fstab) so I’m not the target audience. KTextEditor is missing .editorconfig support. That can come in with the next frameworks update, when consumers update anyway. Chasing it in an intermediate release is a bit problematic because it does require some rebuilds of consumers. Travel Grant Application for BSDCan is now open (https://lists.freebsd.org/pipermail/freebsd-announce/2020-February/001929.html) Hi everyone, The Travel Grant Application for BSDCan 2020 is now open. The Foundation can help you attend BSDCan through our travel grant program. Travel grants are available to FreeBSD developers and advocates who need assistance with travel expenses for attending conferences related to FreeBSD development. BSDCan 2020 applications are due April 9, 2020. Find out more and apply at: https://www.freebsdfoundation.org/what-we-do/grants/travel-grants/ Did you know the Foundation also provides grants for technical events not specifically focused on BSD? If you feel that your attendance at one of these events will benefit the FreeBSD Project and Community and you need assistance getting there, please fill out the general travel grant application. Your application must be received 7 weeks prior to the event. The general application can be found here: https://goo.gl/forms/QzsOMR8Jra0vqFYH2 Creating a ZFS dataset for testing iocage within a jail (https://dan.langille.org/2020/02/01/creating-a-zfs-dataset-for-testing-iocage-within-a-jail/) Be warned, this failed. I’m stalled and I have not completed this. I’m going to do jails within a jail. I already do that with poudriere in a jail but here I want to test an older version of iocage before upgrading my current jail hosts to a newer version. In this post: FreeBSD 12.1 py36-iocage-1.2_3 py36-iocage-1.2_4 This post includes my errors and mistakes. Perhaps you should proceed carefully and read it all first. Beastie Bits Reminder: the FreeBSD Journal is free! Check out these great articles (https://www.freebsdfoundation.org/journal/browser-based-edition/) Serenity GUI desktop running on an OpenBSD kernel (https://twitter.com/jcs/status/1224205573656322048) The Open Source Parts of MacOS (https://github.com/apple-open-source/macos) FOSDEM videos available (https://www.fosdem.org/2020/schedule/track/bsd/) Feedback/Questions Michael - Install with ZFS (http://dpaste.com/3WRC9CQ#wrap) Mohammad - Server Freeze (http://dpaste.com/3BYZKMS#wrap) Todd - ZFS Questions (http://dpaste.com/2J50HSJ#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 20 Feb 2020 05:00:00 -0800
337: Kubernetes on bhyve
Happinesses and stresses of full-time FOSS work, building a FreeBSD fileserver, Kubernetes on FreeBSD bhyve, NetBSD 9 RC1 available, OPNSense 20.1 is here, HardenedBSD’s idealistic future, and more. Headlines The happinesses and stresses of full-time FOSS work (https://drewdevault.com//2020/01/21/Stress-and-happiness.html) In the past few days, several free software maintainers have come out to discuss the stresses of their work. Though the timing was suggestive, my article last week on the philosophy of project governance was, at best, only tangentially related to this topic - I had been working on that article for a while. I do have some thoughts that I’d like to share about what kind of stresses I’ve dealt with as a FOSS maintainer, and how I’ve managed (or often mismanaged) it. February will mark one year that I’ve been working on self-directed free software projects full-time. I was planning on writing an optimistic retrospective article around this time, but given the current mood of the ecosystem I think it would be better to be realistic. In this stage of my career, I now feel at once happier, busier, more fulfilled, more engaged, more stressed, and more depressed than I have at any other point in my life. The good parts are numerous. I’m able to work on my life’s passions, and my projects are in the best shape they’ve ever been thanks to the attention I’m able to pour into them. I’ve also been able to do more thoughtful, careful work; with the extra time I’ve been able to make my software more robust and reliable than it’s ever been. The variety of projects I can invest my time into has also increased substantially, with what was once relegated to minor curiosities now receiving a similar amount of attention as my larger projects were receiving in my spare time before. I can work from anywhere in the world, at any time, not worrying about when to take time off and when to put my head down and crank out a lot of code. The frustrations are numerous, as well. I often feel like I’ve bit off more than I can chew. This has been the default state of affairs for me for a long time; I’m often neglecting half of my projects in order to obtain progress by leaps and bounds in just a few. Working on FOSS full-time has cast this model’s disadvantages into greater relief, as I focus on a greater breadth of projects and spend more time on them. Building a FreeBSD File Server (https://www.vmwareblog.org/building-freebsd-file-server/) Recently at my job, I was faced with a task to develop a file server explicitly suited for the requirements of the company. Needless to say, any configuration of a kind depends on what the infrastructure needs. So, drawing from my personal experience and numerous materials on the web, I came up with the combination FreeBSD+SAMBA+AD as the most appropriate. It appears to be a perfect choice for this environment, and harmonic addition to the existing network configuration since FreeBSD + SAMBA + AD enables admins with the broad range of possibilities for access control. However, as nothing is perfect, this configuration isn’t the best choice if your priority is data protection because it won’t be able to reach the necessary levels of reliability and fault tolerance without outside improvements. Now, since we’ve established that, let’s move on to the next point. This article’s describing the process of building a test environment while concentrating primarily on the details of the configuration. As the author, though, I must say I’m in no way suggesting that this is the only way! The following configuration will be presented in its initial stage, with the minimum requirements necessary to get the job done, and its purpose in one specific situation only. Here, look at this as a useful strategy to solve similar tasks. Well, let’s get started! Report from the first Hamilton BSD Users Group Meeting (https://twitter.com/hambug_ca/status/1227664949914349569) February 11th was the first meeting of this new user group, founded by John Young and myself 11 people attended, and a lot of good discussions were had One of the attendees already owns a domain that fits well for the group, so we will be getting that setup over the next few weeks, as well as the twitter account, and other organization stuff. Special thanks to the illumos users who drove in from Buffalo to attend, although they may have actually had a shorter drive than a few of the other attendees. The next meeting is scheduled again for the 2nd Tuesday of the month, March 10th. We are still discussing if we should meet at a restaurant again, or try to get a space at the local college or innovation hub where we can have a projector etc. News Roundup Kubernetes on FreeBSD Bhyve (https://www.bsdstore.ru/en/articles/cbsd_k8s_part1.html) There are quite a few solutions for container orchestration, but the most popular (or the most famous and highly advertised, is probably, a Kubernetes) Since I plan to conduct many experiments with installing and configuring k8s, I need a laboratory in which I can quickly and easily deploy a cluster in any quantities for myself. In my work and everyday life I use two OS very tightly - Linux and FreeBSD OS. Kubernetes and docker are Linux-centric projects, and at first glance, you should not expect any useful participation and help from FreeBSD here. As the saying goes, an elephant can be made out of a fly, but it will no longer fly. However, two tempting things come to mind - this is very good integration and work in the FreeBSD ZFS file system, from which it would be nice to use the snapshot mechanism, COW and reliability. And the second is the bhyve hypervisor, because we still need the docker and k8s loader in the form of the Linux kernel. Thus, we need to connect a certain number of actions in various ways, most of which are related to starting and pre-configuring virtual machines. This is typical of both a Linux-based server and FreeBSD. What exactly will work under the hood to run virtual machines does not play a big role. And if so - let's take a FreeBSD here! NetBSD 9 RC1 Available (http://blog.netbsd.org/tnf/entry/first_release_candidate_for_netbsd) We hope this will lead to the best NetBSD release ever (only to be topped by NetBSD 10 next year). Here are a few highlights of the new release: Support for Arm AArch64 (64-bit Armv8-A) machines, including "Arm ServerReady" compliant machines (SBBR+SBSA) Enhanced hardware support for Armv7-A Updated GPU drivers (e.g. support for Intel Kabylake) Enhanced virtualization support Support for hardware-accelerated virtualization (NVMM) Support for Performance Monitoring Counters Support for Kernel ASLR Support several kernel sanitizers (KLEAK, KASAN, KUBSAN) Support for userland sanitizers Audit of the network stack Many improvements in NPF Updated ZFS Reworked error handling and NCQ support in the SATA subsystem Support a common framework for USB Ethernet drivers (usbnet) You can download binaries of NetBSD 9.0RC1 from our Fastly-provided CDN: https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.0RC1/ OPNsense 20.1 Keen Kingfisher released (https://opnsense.org/opnsense-20-1-keen-kingfisher-released/) For over 5 years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing. 20.1, nicknamed "Keen Kingfisher", is a subtle improvement on sustainable firewall experience. This release adds VXLAN and additional loopback device support, IPsec public key authentication and elliptic curve TLS certificate creation amongst others. Third party software has been updated to their latest versions. The logging frontend was rewritten for MVC with seamless API support. On the far side the documentation increased in quality as well as quantity and now presents itself in a familiar menu layout. Idealistic Future for HardenedBSD (https://hardenedbsd.org/article/shawn-webb/2020-01-26/idealistic-future-hardenedbsd) Over the past month, we purchased and deployed the new 13-CURRENT/amd64 package building server. We published our first 13-CURRENT/amd64 production package build using that server. We then rebuilt the old package building server to act as the 12-STABLE/amd64 package building server. This post signifies a very important milestone: we have now fully recovered from last year's death of our infrastructure. Our 12-STABLE/amd64 repo, previously out-of-date by many months, is now fully up-to-date! HardenedBSD is in a very unique position to provide innovative solutions to at-risk and underprivileged populations. As such, we are making human rights endeavors a defining area of focus. Our infrastructure will integrate various privacy and anonymity enhancing technologies and techniques to protect lives. Our operating system's security posture will increase, especially with our focus on exploit mitigations. Navigating the intersection between human rights and information security directly impacts lives. HardenedBSD's 2020 mission and focus is to deliver an entire hardened ecosystem that is unfriendly towards those who would oppress or censor their people. This includes a subtle shift in priorities to match this new mission and focus. While we implement exploit mitigations and further harden the ecosystem, we will seek out opportunities to contribute a tangible and unique impact on human rights issues. Providing Tor Onion Services for our core infrastructure is the first step in likely many to come towards securely helping those in need. Beastie Bits Warner Losh's FOSDEM talk (https://fosdem.org/2020/interviews/warner-losh/) Relational Pipes v0.15 (https://relational-pipes.globalcode.info/v_0/release-v0.15.xhtml) A reminder for where to find NetBSD ARM images (http://www.armbsd.org/arm/) New Safe Memory Reclamation feature in UMA (https://lists.freebsd.org/pipermail/freebsd-arch/2020-January/019866.html) BSD Users Stockholm Meetup (https://twitter.com/niclaszeising/status/1216667359831842817) Feedback/Questions ZFS - Rosetta Stone Document? (http://dpaste.com/13EK8YH#wrap) Pat - Question (http://dpaste.com/2DN5RA4#wrap) Sigflup - Wayland on the BSDs (http://dpaste.com/03Y4FQ7#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 13 Feb 2020 08:30:00 -0800
336: Archived Knowledge
Linux couldn’t duplicate OpenBSD, FreeBSD Q4 status report, OPNsense 19.7.9 released, archives retain and pass on knowledge, HardenedBSD Tor Onion Service v3 Nodes, and more. Headlines OpenBSD has to be a BSD Unix and you couldn't duplicate it with Linux (https://utcc.utoronto.ca/~cks/space/blog/unix/OpenBSDMustBeABSD?showcomments) OpenBSD has a well deserved reputation for putting security and a clean system (for code, documentation, and so on) first, and everything else second. OpenBSD is of course based on BSD (it's right there in the name) and descends from FreeBSD NetBSD (you can read the history here). But one of the questions you could ask about it is whether it had to be that way, and in particular if you could build something like OpenBSD on top of Linux. I believe that the answer is no. Linux and the *BSDs have a significantly different model of what they are. BSDs have a 'base system' that provides an integrated and fully operational core Unix, covering the kernel, C library and compiler, and the normal Unix user level programs, all maintained and distributed by the particular BSD. Linux is not a single unit this way, and instead all of the component parts are maintained separately and assembled in various ways by various Linux distributions. Both approaches have their advantages, but one big one for the BSD approach is that it enables global changes. Making global changes is an important part of what makes OpenBSD's approach to improving security, code maintenance, and so on work. Because it directly maintains everything as a unit, OpenBSD is in a position to introduce new C library or kernel APIs (or change them) and then immediately update all sorts of things in user level programs to use the new API. This takes a certain amount of work, of course, but it's possible to do it at all. And because OpenBSD can do this sort of ambitious global change, it does. This goes further than just the ability to make global changes, because in theory you can patch in global changes on top of a bunch of separate upstream projects. Because OpenBSD is in control of its entire base system, it's not forced to try to reconcile different development priorities or integrate clashing changes. OpenBSD can decide (and has) that only certain sorts of changes will be accepted into its system at all, no matter what people want. If there are features or entire programs that don't fit into what OpenBSD will accept, they just lose out. FreeBSD Quarterly Status Report 2019Q4 (https://lists.freebsd.org/pipermail/freebsd-announce/2020-January/001923.html) Here is the last quarterly status report for 2019. As you might remember from last report, we changed our timeline: now we collect reports the last month of each quarter and we edit and publish the full document the next month. Thus, we cover here the period October 2019 - December 2019. If you thought that the FreeBSD community was less active in the Christmas' quarter you will be glad to be proven wrong: a quick glance at the summary will be sufficient to see that much work has been done in the last months. Have a nice read! News Roundup OPNsense 19.7.9 released (https://opnsense.org/opnsense-19-7-9-released/) As 20.1 nears we will be making adjustments to the scope of the release with an announcement following shortly. For now, this update brings you a GeoIP database configuration page for aliases which is now required due to upstream database policy changes and a number of prominent third-party software updates we are happy to see included. Archives are important to retain and pass on knowledge (https://dan.langille.org/2020/01/07/archives-are-important-to-retain-and-pass-on-knowledge/) Archives are important. When they are public and available for searching, it retains and passes on knowledge. It saves vast amounts of time. HardenedBSD Tor Onion Service v3 Nodes (https://hardenedbsd.org/article/shawn-webb/2020-01-30/hardenedbsd-tor-onion-service-v3-nodes) I've been working today on deploying Tor Onion Service v3 nodes across our build infrastructure. I'm happy to announce that the public portion of this is now completed. Below you will find various onion service hostnames and their match to our infrastructure. hardenedbsd.org: lkiw4tmbudbr43hbyhm636sarn73vuow77czzohdbqdpjuq3vdzvenyd.onion ci-01.nyi.hardenedbsd.org: qspcqclhifj3tcpojsbwoxgwanlo2wakti2ia4wozxjcldkxmw2yj3yd.onion ci-03.md.hardenedbsd.org: eqvnohly4tjrkpwatdhgptftabpesofirnhz5kq7jzn4zd6ernpvnpqd.onion ci-04.md.hardenedbsd.org: rfqabq2w65nhdkukeqwf27r7h5xfh53h3uns6n74feeyl7s5fbjxczqd.onion git-01.md.hardenedbsd.org: dacxzjk3kq5mmepbdd3ai2ifynlzxsnpl2cnkfhridqfywihrfftapid.onion Beastie Bits The Missing Semester of Your CS Education (MIT Course) (https://missing.csail.mit.edu/) An old Unix Ad (https://i.redd.it/503390rf7md41.png) OpenBSD syscall call-from verification (https://marc.info/?l=openbsd-tech&m=157488907117170&w=2) OpenBSD/arm64 on Pinebook (https://twitter.com/bluerise/status/1220963106563579909) Reminder: First Southern Ontario BSD user group meeting, February 11th (this coming Tuesday!) 18:30 at Boston Pizza on Upper James st, Hamilton. (http://studybsd.com/) NYCBUG: March meeting will feature Dr. Paul Vixie and his new talk “Operating Systems as Dumb Pipes” (https://www.nycbug.org/) 8th Meetup of the Stockholm BUG: March 3 at 18:00 (https://www.meetup.com/de-DE/BSD-Users-Stockholm/events/267873938/) Polish BSD User Group meets on Feb 11, 2020 at 18:15 (https://bsd-pl.org/en) Feedback/Questions Sean - ZFS and Creation Dates (http://dpaste.com/3W5WBV0#wrap) Christopher - Help on ZFS Disaster Recovery (http://dpaste.com/3SE43PW) Mike - Encrypted ZFS Send (http://dpaste.com/00J5JZG#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 06 Feb 2020 05:00:00 -0800
335: FreeBSD Down Under
Hyperbola Developer interview, why you should migrate from Linux to BSD, FreeBSD is an amazing OS, improving the ptrace(2) API in LLVM 10, First FreeBSD conference in Australia, and a guide to containers on FreeNAS. Headlines FreeBSD is an amazing operating System (https://www.unixsheikh.com/articles/freebsd-is-an-amazing-operating-system.html) Update 2020-01-21: Since I wrote this article it got posted on Hacker News, Reddit and Lobster, and a few people have emailed me with comments. I have updated the article with comments where I have found it needed. As an important side note I would like to point out that I am not a FreeBSD developer, there may be things going on in the FreeBSD world that I know absolutely nothing about. I am also not glued to the FreeBSD developer mailing lists. I am not a FreeBSD "fanboy". I have been using GNU/Linux a ton more for the past two decades than FreeBSD, mainly due to hardware incompatibility (lacking or buggy drivers), and I love both Debian GNU/Linux and Arch Linux just as much as FreeBSD. However, I am concerned about the development of GNU/Linux as of late. Also this article is not about me trying to make anyone switch from something else to FreeBSD. It's about why I like FreeBSD and that I recommend you try it out if you're into messing with operating systems. I think the year was late 1999 or mid 2000 when I one day was browsing computer books at my favorite bookshop and I discovered the book The Complete FreeBSD third edition from 1999 by Greg Lehey. With the book came 4 CD Roms with FreeBSD 3.3. I had already familiarized myself with GNU/Linux in 1998, and I was in the process of migrating every server and desktop operating system away from Microsoft Windows, both at home and at my company, to GNU/Linux, initially Red Hat Linux and then later Debian GNU/Linux, which eventually became my favorite GNU/Linux distribution for many years. When I first saw The Complete FreeBSD book by Greg Lehey I remember noticing the text on the front page that said, "The Free Version of Berkeley UNIX" and "Rock Solid Stability", and I was immediately intrigued! What was that all about? A free UNIX operating system! And rock solid stability? That sounded amazing. Hyperbola Dev Interview (https://itsfoss.com/hyperbola-linux-bsd/) In late December 2019, Hyperbola announced that they would be making major changes to their project. They have decided to drop the Linux kernel in favor of forking the OpenBSD kernel. This announcement only came months after Project Trident announced that they were going in the opposite direction (from BSD to Linux). Hyperbola also plans to replace all software that is not GPL v3 compliant with new versions that are. To get more insight into the future of their new project, I interviewed Andre, co-founder of Hyperbola. News Roundup Improving the ptrace(2) API and preparing for LLVM-10.0 (https://blog.netbsd.org/tnf/entry/improving_the_ptrace_2_api) This month I have improved the NetBSD ptrace(2) API, removing one legacy interface with a few flaws and replacing it with two new calls with new features, and removing technical debt. As LLVM 10.0 is branching now soon (Jan 15th 2020), I worked on proper support of the LLVM features for NetBSD 9.0 (today RC1) and NetBSD HEAD (future 10.0). The first FreeBSD conference in Australia (https://rubenerd.com/the-first-freebsd-conference-in-australia/) FreeBSD has existed as an operating system, project, and foundation for more than twenty years, and its earlier incantations have exited for far longer. The old guard have been developing code, porting software, and writing documentation for longer than I’ve existed. I’ve been using it for more than a decade for personal projects, and professionally for half that time. While there are many prominent Australian FreeBSD contributors, sysadmins, and users, we’ve always had to venture overseas for conferences. We’re always told Australians are among the most ardent travellers, but I always wondered if we could do a domestic event as well. And on Tuesday, we did! Deb Goodkin and the FreeBSD Foundation graciously organised and chaired a dedicated FreeBSD miniconf at the long-running linux.conf.au event held each year in a different city in Australia and New Zealand. A practical guide to containers on FreeNAS for a depraved psychopath (https://medium.com/@andoriyu/a-practical-guide-to-containers-on-freenas-for-a-depraved-psychopath-c212203c0394) This is a simple write-up to setup Docker on FreeNAS 11 or FreeBSD 11. But muh jails? You know that jails are dope and you know that jails are dope, yet no one else knows it. So here we are stuck with docker. Two years ago I would be the last person to recommend using docker, but a whole lot of things has changes past years… So jails are dead then? No, jails are still dope, but jails lack tools to manage them. Yes, there are a few tools, but they meant for hard-core FreeBSD users who used to suffering. Docker allows you to run applications without deep knowledge of application you’re running. It will also allow you to run applications that are not ported to FreeBSD. Why you should migrate everything from Linux to BSD (https://www.unixsheikh.com/articles/why-you-should-migrate-everything-from-linux-to-bsd.html) As an operating system GNU/Linux has become a real mess because of the fragmented nature of the project, the bloatware in the kernel, and because of the jerking around by commercial interests. Response Should you migrate from Linux to BSD? It depends. (https://fediverse.blog/~/AllGoodThings/should-you-migrate-from-linux-to-bsd-it-depends) Beastie Bits Using the OpenBSD ports tree with dedicated users (https://dataswamp.org/~solene/2020-01-11-privsep.html) broot on FreeBSD (https://vermaden.wordpress.com/2020/01/10/run-broot-on-freebsd/) A Trip down Memory Lane (https://svnweb.freebsd.org/base/head/share/misc/bsd-family-tree?view=co) Running syslog-ng in BastilleBSD (https://www.syslog-ng.com/community/b/blog/posts/running-syslog-ng-in-bastillebsd) NASA : Using Software Packages in pkgsrc (https://www.nas.nasa.gov/hecc/support/kb/using-software-packages-in-pkgsrc_493.html) Feedback/Questions All of our questions this week were pretty technical in nature so I'm going to save those for the next episode so Allan can weigh in on them, since if we cover them now we're basically going to be deferring to Allan anyway. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 30 Jan 2020 05:00:00 -0800
334: Distrowatch Running FreeBSD
Upgrading FreeBSD from 11.3 to 12.1, Distrowatch switching to FreeBSD, Torvalds says don’t run ZFS, iked(8) removed automatic IPv6 blocking, working towards LLDB on i386, and memory-hard Argon2 hashing scheme in NetBSD. Headlines Upgrading FreeBSD from 11.3 to 12.1 (https://blog.bimajority.org/2020/01/13/upgrading-freebsd-from-11-3-to-12-1/) Now here’s something more like what I was originally expecting the content on this blog to look like. I’m in the process of moving all of our FreeBSD servers (about 30 in total) from 11.3 to 12.1. We have our own local build of the OS, and until “packaged base” gets to a state where it’s reliably usable, we’re stuck doing upgrades the old-fashioned way. I created a set of notes for myself while cranking through these upgrades and I wanted to share them since they are not really work-specific and this process isn’t very well documented for people who haven’t been doing this sort of upgrade process for 25 years. Our source and object trees are read-only exported from the build server over NFS, which causes things to be slow. /etc/make.conf and /etc/src.conf are symbolic links on all of our servers to the master copies in /usr/src so that make installworld can find the configuration parameters the system was built with. Switching Distrowatch over to BSD (https://www.reddit.com/r/freebsd/comments/eodhit/switching_distrowatch_over_to_freebsd_ama/) This may be a little off-topic for this board (forgive me if it is, please). However, I wanted to say that I'm one of the people who works on DistroWatch (distrowatch.com) and this past week we had to deal with a server facing hardware failure. We had a discussion about whether to continue running Debian or switch to something else. The primary "something else" option turned out to be FreeBSD and it is what we eventually went with. It took a while to convert everything over from working with Debian GNU/Linux to FreeBSD 12 (some script incompatibilities, different paths, some changes to web server configuration, networking IPv6 troubles). But in the end we ended up with a good, FreeBSD-based experience. Since the transition was successful, though certainly not seamless, I thought people might want to do a Q&A on the migration process. Especially for those thinking of making the same switch. News Roundup iked(8) automatic IPv6 blocking removed (https://www.openbsd.org/faq/current.html#r20200114) iked(8) no longer automatically blocks unencrypted outbound IPv6 packets. This feature was intended to avoid accidental leakage, but in practice was found to mostly be a cause of misconfiguration. If you previously used iked(8)'s -6 flag to disable this feature, it is no longer needed and should be removed from /etc/rc.conf.local if used. Linus says dont run ZFS (https://itsfoss.com/linus-torvalds-zfs/) “Don’t use ZFS. It’s that simple. It was always more of a buzzword than anything else, I feel, and the licensing issues just make it a non-starter for me.” This is what Linus Torvalds said in a mailing list to once again express his disliking for ZFS filesystem specially over its licensing. To avoid unnecessary confusion, this is more intended for Linux distributions, kernel developers and maintainers rather than individual Linux users. GSoC 2019 Final Report: Incorporating the memory-hard Argon2 hashing scheme into NetBSD (https://blog.netbsd.org/tnf/entry/gsoc_2019_final_report_incorporating) We successfully incorporated the Argon2 reference implementation into NetBSD/amd64 for our 2019 Google Summer of Coding project. We introduced our project here and provided some hints on how to select parameters here. For our final report, we will provide an overview of what changes were made to complete the project. The Argon2 reference implementation, available here, is available under both the Creative Commons CC0 1.0 and the Apache Public License 2.0. To import the reference implementation into src/external, we chose to use the Apache 2.0 license for this project. Working towards LLDB on i386 NetBSD (https://blog.netbsd.org/tnf/entry/working_towards_lldb_on_i386) Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February 2019, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, fixing watchpoint and threading support. Throughout December I've continued working on our build bot maintenance, in particular enabling compiler-rt tests. I've revived and finished my old patch for extended register state (XState) in core dumps. I've started working on bringing proper i386 support to LLDB. Beastie Bits An open source Civilization V (https://github.com/yairm210/UnCiv) BSD Groups in Italy (https://bsdnotizie.blogspot.com/2020/01/gruppi-bsd-in-italia.html) Why is Wednesday, November 17, 1858 the base time for OpenVMS? (https://www.slac.stanford.edu/~rkj/crazytime.txt) Benchmarking shell pipelines and the Unix “tools” philosophy (https://blog.plover.com/Unix/tools.html) LPI and BSD working together (https://youtu.be/QItb5aoj7Oc) Feedback/Questions Pat - March Meeting (http://dpaste.com/2BMGZVV#wrap) Madhukar - Overheating Laptop (http://dpaste.com/17WNVM8#wrap) Warren - R vs S (http://dpaste.com/3AZYFB1#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 23 Jan 2020 05:00:00 -0800
333: Unix Keyboard Joy
Your Impact on FreeBSD in 2019, Wireguard on OpenBSD Router, Amazon now has FreeBSD/ARM 12, pkgsrc-2019Q4, The Joys of UNIX Keyboards, OpenBSD on Digital Ocean, and more. Headlines Your Impact on FreeBSD in 2019 (https://www.freebsdfoundation.org/blog/your-impact-on-freebsd-in-2019/) It’s hard to believe that 2019 is nearly over. It has been an amazing year for supporting the FreeBSD Project and community! Why do I say that? Because as I reflect over the past 12 months, I realize how many events we’ve attended all over the world, and how many lives we’ve touched in so many ways. From advocating for FreeBSD to implementing FreeBSD features, my team has been there to help make FreeBSD the best open source project and operating system out there. In 2019, we focused on supporting a few key areas where the Project needed the most help. The first area was software development. Whether it was contracting FreeBSD developers to work on projects like wifi support, to providing internal staff to quickly implement hardware workarounds, we’ve stepped in to help keep FreeBSD innovative, secure, and reliable. Software development includes supporting the tools and infrastructure that make the development process go smoothly, and we’re on it with team members heading up the Continuous Integration efforts, and actively involved in the clusteradmin and security teams. Our advocacy efforts focused on recruiting new users and contributors to the Project. We attended and participated in 38 conferences and events in 21 countries. From giving FreeBSD presentations and workshops to staffing tables, we were able to have 1:1 conversations with thousands of attendees. Our travels also provided opportunities to talk directly with FreeBSD commercial and individual users, contributors, and future FreeBSD user/contributors. We’ve seen an increase in use and interest in FreeBSD from all of these organizations and individuals. These meetings give us a chance to learn more about what organizations need and what they and other individuals are working on. The information helps inform the work we should fund. Wireguard on OpenBSD Router (https://obscurity.xyz/bsd/open/wireguard.html) wireguard (wg) is a modern vpn protocol, using the latest class of encryption algorithms while at the same time promising speed and a small code base. modern crypto and lean code are also tenants of openbsd, thus it was a no brainer to migrate my router from openvpn over to wireguard. my setup : a collection of devices, both wired and wireless, that are nat’d through my router (openbsd 6.6) out via my vpn provider azire* and out to the internet using wg-quick to start wg. running : doubtless this could be improved on, but currently i start wg manually when my router boots. this, and the nat'ing on the vpn interface mean its impossible for clients to connect to the internet without the vpn being up. as my router is on a ups and only reboots when a kernel patch requires it, it’s a compromise i can live with. run wg-quick (please replace vpn with whatever you named your wg .conf file.) and reload pf rules. News Roundup Amazon now has FreeBSD/ARM 12 (https://aws.amazon.com/marketplace/pp/B081NF7BY7) AWS, the cloud division of Amazon, announced in December the next generation of its ARM processors, the Graviton2. This is a custom chip design with a 7nm architecture. It is based on 64-bit ARM Neoverse cores. Compared to first-generation Graviton processors (A1), today’s new chips should deliver up to 7x the performance of A1 instances in some cases. Floating point performance is now twice as fast. There are additional memory channels and cache speed memory access should be much faster. The company is working on three types of Graviton2 EC2 instances that should be available soon. Instances with a “g” suffix are powered by Graviton2 chips. If they have a “d” suffix, it also means that they have NVMe local storage. General-purpose instances (M6g and M6gd) Compute-optimized instances (C6g and C6gd) Memory-optimized instances (R6g and R6gd) You can choose instances with up to 64 vCPUs, 512 GiB of memory and 25 Gbps networking. And you can see that ARM-powered servers are not just a fad. AWS already promises a 40% better price/performance ratio with ARM-based instances when you compare them with x86-based instances. AWS has been working with operating system vendors and independent software vendors to help them release software that runs on ARM. ARM-based EC2 instances support Amazon Linux 2, Ubuntu, Red Hat, SUSE, Fedora, Debian and FreeBSD. It also works with multiple container services (Docker, Amazon ECS, and Amazon Elastic Kubernetes Service). Coverage of AWS Announcement (https://techcrunch.com/2019/12/03/aws-announces-new-arm-based-instances-with-graviton2-processors/) Announcing the pkgsrc-2019Q4 release (https://mail-index.netbsd.org/pkgsrc-users/2020/01/06/msg030130.html) The pkgsrc developers are proud to announce the 65th quarterly release of pkgsrc, the cross-platform packaging system. pkgsrc is available with more than 20,000 packages, running on 23 separate platforms; more information on pkgsrc itself is available at https://www.pkgsrc.org/ In total, 190 packages were added, 96 packages were removed, and 1,868 package updates (to 1388 unique packages) were processed since the pkgsrc-2019Q3 release. As usual, a large number of updates and additions were processed for packages for go (14), guile (11), perl (170), php (10), python (426), and ruby (110). This continues pkgsrc's tradition of adding useful packages, updating many packages to more current versions, and pruning unmaintained packages that are believed to have essentially no users. The Joys of UNIX Keyboards (https://donatstudios.com/UNIX-Keyboards) I fell in love with a dead keyboard layout. A decade or so ago while helping a friends father clean out an old building, we came across an ancient Sun Microsystems server. We found it curious. Everything about it was different from what we were used to. The command line was black on white, the connectors strange and foreign, and the keyboard layout was bizarre. We never did much with it; turning it on made all the lights in his home dim, and our joint knowledge of UNIX was nonexistent. It sat in his bedroom for years supporting his television at the foot of his bed. I never forgot that keyboard though. The thought that there was this alternative layout out there seemed intriguing to me. OpenBSD on Digital Ocean (https://www.going-flying.com/blog/openbsd-on-digitalocean.html) Last night I had a need to put together a new OpenBSD machine. Since I already use DigitalOcean for one of my public DNS servers I wanted to use them for this need but sadly like all too many of the cloud providers they don't support OpenBSD. Now they do support FreeBSD and I found a couple writeups that show how to use FreeBSD as a shim to install OpenBSD. They are both sort of old at this point and with OpenBSD 6.6 out I ran into a bit of a snag. The default these days is to use a GPT partition table to enable EFI booting. This is generally pretty sane but it looks to me like the FreeBSD droplet doesn't support this. After the installer rebooted the VM failed to boot, being unable to find the bootloader. Thankfully DigitalOcean has a recovery ISO that you can boot by simply switching to it and powering off and then on your Droplet. Beastie Bits FreeBSD defaults to LLVM on PPC (https://svnweb.freebsd.org/base?view=revision&revision=356111) Theo De Raadt Interview between Ottawa 2019 Hackathon and BSDCAN 2019 (https://undeadly.org/cgi?action=article;sid=20191231214356) Bastille Poll about what people would like to see in 2020 (https://twitter.com/BastilleBSD/status/1211475103143251968) Notes on the classic book : The Design of the UNIX Operating System (https://github.com/suvratapte/Maurice-Bach-Notes) Multics History (https://www.multicians.org/) First meeting of the Hamilton BSD user group, February 11, 2020 18:30 - 21:00, Boston Pizza on Upper James St (http://studybsd.com/) Feedback/Questions Bill - 1.1 CDROM (http://dpaste.com/2H9CW6R) Greg - More 50 Year anniversary information (http://dpaste.com/2SGA3KY) Dave - Question time for Allan (http://dpaste.com/3ZAEKHD#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 16 Jan 2020 05:00:00 -0800
332: The BSD Hyperbole
Announcing HyperbolaBSD, IPFW In-Kernel NAT setup on FreeBSD, Wayland and WebRTC enabled for NetBSD 9/Linux, LLDB Threading support ready for mainline, OpenSSH U2F/FIDO support in base, Dragonfly drm/i915: Update, and more. Headlines HyperbolaBSD Announcement (https://www.hyperbola.info/news/announcing-hyperbolabsd-roadmap/) Due to the Linux kernel rapidly proceeding down an unstable path, we are planning on implementing a completely new OS derived from several BSD implementations. This was not an easy decision to make, but we wish to use our time and resources to create a viable alternative to the current operating system trends which are actively seeking to undermine user choice and freedom. This will not be a "distro", but a hard fork of the OpenBSD kernel and userspace including new code written under GPLv3 and LGPLv3 to replace GPL-incompatible parts and non-free ones. Reasons for this include: Linux kernel forcing adaption of DRM, including HDCP. Linux kernel proposed usage of Rust (which contains freedom flaws and a centralized code repository that is more prone to cyber attack and generally requires internet access to use.) Linux kernel being written without security and in mind. (KSPP is basically a dead project and Grsec is no longer free software) Many GNU userspace and core utils are all forcing adaption of features without build time options to disable them. E.g. (PulseAudio / SystemD / Rust / Java as forced dependencies) As such, we will continue to support the Milky Way branch until 2022 when our legacy Linux-libre kernel reaches End of Life. Future versions of Hyperbola will be using HyperbolaBSD which will have the new kernel, userspace and not be ABI compatible with previous versions. HyperbolaBSD is intended to be modular and minimalist so other projects will be able to re-use the code under free license. Forum Post (https://forums.hyperbola.info/viewtopic.php?id=315) A simple IPFW In-Kernel NAT setup on FreeBSD (https://www.neelc.org/posts/freebsd-ipfw-nat/) After graduating college, I am moving from Brooklyn, NY to Redmond, WA (guess where I got a job). I always wanted to re-do my OPNsense firewall (currently a HP T730) with stock FreeBSD and IPFW’s in-kernel NAT. Why IPFW? Benchmarks have shown IPFW to be faster which is especially good for my Tor relay, and because I can! However, one downside of IPFW is less documentation vs PF, even less without natd (which we’re not using), and this took me time to figure this out. But since my T730 is already packed, I am testing this on a old PC with two NICs, and my laptop [1] as a client with an USB-to-Ethernet adapter. News Roundup HEADS UP: Wayland and WebRTC enabled for NetBSD 9/Linux (https://mail-index.netbsd.org/pkgsrc-users/2020/01/05/msg030124.html) This is just a heads up that the Wayland option is now turned on by default for NetBSD 9 and Linux in cases where it peacefully coexists with X11. Right now, this effects the following packages: graphics/MesaLib devel/SDL2 www/webkit-gtk x11/gtk3 The WebRTC option has also been enabled by default on NetBSD 9 for two Firefox versions: www/firefox, www/firefox68 Please keep me informed of any fallout. Hopefully, there will be none. If you want to try out Wayland-related things on NetBSD 9, wm/velox/MESSAGE may be interesting for you. LLDB Threading support now ready for mainline (https://blog.netbsd.org/tnf/entry/lldb_threading_support_now_ready) Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report. So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report. OpenSSH U2F/FIDO support in base (https://www.undeadly.org/cgi?action=article;sid=20191115064850) Hardware backed keys can be generated using "ssh-keygen -t ecdsa-sk" (or "ed25519-sk" if your token supports it). Many tokens require to be touched/tapped to confirm this step. You'll get a public/private keypair back as usual, except in this case, the private key file does not contain a highly-sensitive private key but instead holds a "key handle" that is used by the security key to derive the real private key at signing time. So, stealing a copy of the private key file without also stealing your security key (or access to it) should not give the attacker anything. drm/i915: Update to Linux 4.8.17 (http://lists.dragonflybsd.org/pipermail/commits/2019-December/720257.html) drm/i915: Update to Linux 4.8.17 Broxton, Valleyview and Cherryview support improvements Broadwell and Gen9/Skylake support improvements Broadwell brightness fixes from OpenBSD Atomic modesetting improvements Various bug fixes and performance enhancements Beastie Bits Visual Studio Code port for FreeBSD (https://github.com/tagattie/FreeBSD-VSCode) OpenBSD syscall call-from verification (https://marc.info/?l=openbsd-tech&m=157488907117170&w=2) Peertube on OpenBSD (https://www.22decembre.eu/en/2019/12/09/peertube-14-openbsd/) Fuzzing Filesystems on NetBSD via AFL+KCOV by Maciej Grochowski (https://www.youtube.com/watch?v=bbNCqFdQEyk&feature=youtu.be) Twitter Bot for Prop65 (https://twitter.com/prop65bot/status/1199003319307558912) Interactive vim tutorial (https://www.openvim.com/) First BSD user group meeting in Hamilton, February 11, 2020 18:30 - 21:00, Boston Pizza on Upper James St (http://studybsd.com/) *** Feedback/Questions Samir - cgit (http://dpaste.com/2B22M24#wrap) Russell - R (http://dpaste.com/0J5TYY0#wrap) Wolfgang - Question (http://dpaste.com/3MQAH27#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 09 Jan 2020 05:00:00 -0800
331: Why Computers Suck
How learning OpenBSD makes computers suck a little less, How Unix works, FreeBSD 12.1 Runs Well on Ryzen Threadripper 3970X, BSDCan CFP, HardenedBSD Infrastructure Goals, and more. Headlines Why computers suck and how learning from OpenBSD can make them marginally less horrible (https://telegra.ph/Why-OpenBSD-is-marginally-less-horrible-12-05) How much better could things actually be if we abandoned the enterprise development model? Next I will compare this enterprise development approach with non-enterprise development - projects such as OpenBSD, which do not hesitate to introduce ABI breaking changes to improve the codebase. One of the most commonly referred to pillars of the project's philosophy has long been its emphasis on clean functional code. Any code which makes it into OpenBSD is subject to ongoing aggressive audits for deprecated, or otherwise unmaintained code in order to reduce cruft and attack surface. Additionally the project creator, Theo de Raadt, and his team of core developers engage in ongoing development for proactive mitigations for various attack classes many of which are directly adopted by various multi-platform userland applications as well as the operating systems themselves (Windows, Linux, and the other BSDs). Frequently it is the case that introducing new features (not just deprecating old ones) introduces new incompatibilities against previously functional binaries compiled for OpenBSD. To prevent the sort of kernel memory bloat that has plagued so many other operating systems for years, the project enforces a hard ceiling on the number of lines of code that can ever be in ring 0 at a given time. Current estimates guess the number of bugs per line of code in the Linux kernel are around 1 bug per every 10,000 lines of code. Think of this in the context of the scope creep seen in the Linux kernel (which if I recall correctly is currently at around 100,000,000 lines of code), as well as the Windows NT kernel (500,000,000 lines of code) and you quickly begin to understand how adding more and more functionality into the most privileged components of the operating system without first removing old components begins to add up in terms of the drastic difference seen between these systems in the number of zero day exploits caught in the wild respectively. How Unix Works: Become a Better Software Engineer (https://neilkakkar.com/unix.html) Unix is beautiful. Allow me to paint some happy little trees for you. I’m not going to explain a bunch of commands – that’s boring, and there’s a million tutorials on the web doing that already. I’m going to leave you with the ability to reason about the system. Every fancy thing you want done is one google search away. But understanding why the solution does what you want is not the same. That’s what gives you real power, the power to not be afraid. And since it rhymes, it must be true. News Roundup FreeBSD 12.1 Runs Refreshingly Well With AMD Ryzen Threadripper 3970X (https://www.phoronix.com/scan.php?page=article&item=freebsd-amd-3970x&num=1) For those of you interested in AMD's new Ryzen Threadripper 3960X/3970X processors with TRX40 motherboards for running FreeBSD, the experience in our initial testing has been surprisingly pleasant. In fact, it works out-of-the-box which one could argue is better than the current Linux support that needs the MCE workaround for booting. Here are some benchmarks of FreeBSD 12.1 on the Threadripper 3970X compared to Linux and Windows for this new HEDT platform. It was refreshing to see FreeBSD 12.1 booting and running just fine with the Ryzen Threadripper 3970X 32-core/64-thread processor from the ASUS ROG ZENITH II EXTREME motherboard and all core functionality working including the PCIe 4.0 NVMe SSD storage, onboard networking, etc. The system was running with 4 x 16GB DDR4-3600 memory, 1TB Corsair Force MP600 NVMe SSD, and Radeon RX 580 graphics. It was refreshing to see FreeBSD 12.1 running well with this high-end AMD Threadripper system considering Linux even needed a boot workaround. While the FreeBSD 12.1 experience was trouble-free with the ASUS TRX40 motherboard (ROG Zenith II Extreme) and AMD Ryzen Threadripper 3970X, DragonFlyBSD unfortunately was not. Both DragonFlyBSD 5.6.2 stable and the DragonFlyBSD daily development snapshot from last week were yielding a panic on boot. So with that, DragonFlyBSD wasn't tested for this Threadripper 3970X comparison but just FreeBSD 12.1. FreeBSD 12.1 on the Threadripper 3970X was benchmarked both with its default LLVM Clang 8.0.1 compiler and again with GCC 9.2 from ports for ruling out compiler differences. The FreeBSD 12.1 performance was compared to last week's Windows 10 vs. Linux benchmarks with the same system. BSDCan 2020 CFP (https://lists.bsdcan.org/pipermail/bsdcan-announce/2019-December/000180.html) BSDCan 2020 will be held 5-6 (Fri-Sat) June, 2020 in Ottawa, at the University of Ottawa. It will be preceded by two days of tutorials on 3-4 June (Wed-Thu). NOTE the change of month in 2020 back to June Also: do not miss out on the Goat BOF on Tuesday 2 June. We are now accepting proposals for talks. The talks should be designed with a very strong technical content bias. Proposals of a business development or marketing nature are not appropriate for this venue. See http://www.bsdcan.org/2020/ If you are doing something interesting with a BSD operating system, please submit a proposal. Whether you are developing a very complex system using BSD as the foundation, or helping others and have a story to tell about how BSD played a role, we want to hear about your experience. People using BSD as a platform for research are also encouraged to submit a proposal. Possible topics include: How we manage a giant installation with respect to handling spam. and/or sysadmin. and/or networking. Cool new stuff in BSD Tell us about your project which runs on BSD other topics (see next paragraph) From the BSDCan website, the Archives section will allow you to review the wide variety of past BSDCan presentations as further examples. Both users and developers are encouraged to share their experiences. HardenedBSD Infrastructure Goals (https://github.com/lattera/articles/blob/master/hardenedbsd/2019-12-01_infrastructure/article.md) 2019 has been an extremely productive year with regards to HardenedBSD's infrastructure. Several opportunities aligned themselves in such a way as to open a door for a near-complete rebuild with a vast expansion. The last few months especially have seen a major expansion of our infrastructure. We obtained a number of to-be-retired Dell R410 servers. The crash of our nightly build server provided the opportunity to deploy these R410 servers, doubling our build capacity. My available time to spend on HardenedBSD has decreased compared to this time last year. As part of rebuilding our infrastructure, I wanted to enable the community to be able to contribute. I'm structuring the work such that help is just a pull request away. Those in the HardenedBSD community who want to contribute to the infrastructure work can simply open a pull request. I'll review the code, and deploy it after a successful review. Users/contributors don't need access to our servers in order to improve them. My primary goal for the rest of 2019 and into 2020 is to become fully self-hosted, with the sole exception of email. I want to transition the source-of-truth git repos to our own infrastructure. We will still provide a read-only mirror on GitHub. As I develop this infrastructure, I'm doing so with human rights in mind. HardenedBSD is in a very unique position. In 2020, I plan to provide production Tor Onion Services for the various bits of our infrastructure. HardenedBSD will provide access to its various internal services to its developers and contributors. The entire development lifecycle, going from dev to prod, will be able to happen over Tor. Transparency will be key moving forward. Logs for the auto-sync script are now published directly to GitHub. Build logs will be, soon, too. Logs of all automated processes, and the code for those processes, will be tracked publicly via git. This will be especially crucial for development over Tor. Integrating Tor into our infrastructure so deeply increases risk and maintenance burden. However, I believe that through added transparency, we will be able to mitigate risk. Periodic audits will need to be performed and published. I hope to migrate HardenedBSD's site away from Drupal to a static site generator. We don't really need the dynamic capabilities Drupal gives us. The many security issues Drupal and PHP both bring also leave much to be desired. So, that's about it. I spent the last few months of 2019 laying the foundation for a successful 2020. I'm excited to see how the project grows. Beastie Bits FuryBSD - KDE plasma flavor now available (https://www.furybsd.org/kde-plasma-flavor-now-available/) DragonFly - git: virtio - Fix LUN scan issue w/ Google Cloud (http://lists.dragonflybsd.org/pipermail/commits/2019-November/719945.html) LPI is looking for BSD Specialist learning material writers (https://wiki.lpi.org/wiki/BSD_Specialist_Objectives_V1.0) ZFS sync/async + ZIL/SLOG, explained (https://jrs-s.net/2019/05/02/zfs-sync-async-zil-slog/) BSD-Licensed Combinatorics library/utility (https://lists.freebsd.org/pipermail/freebsd-announce/2019-December/001921.html) SSL client vs server certificates and bacula-fd (https://dan.langille.org/2019/11/29/ssl-client-vs-server-certificates-and-bacula-fd/) MaxxDesktop planning to come to FreeBSD (https://www.facebook.com/maxxdesktop/posts/2761326693888282) Project Page (https://www.facebook.com/maxxdesktop/) Feedback/Questions Tom - ZFS Mirror with different speeds (http://dpaste.com/3ZGYNS3#wrap) Jeff - Knowledge is power (http://dpaste.com/1H9QDCR#wrap) Johnny - Episode 324 response to Jacob (http://dpaste.com/1A7Q9EV) Pat - NYC*BUG meeting Jan Meeting Location (http://dpaste.com/0QPZ2GC) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 02 Jan 2020 05:00:00 -0800
330: Happy Holidays, All(an)
Authentication Vulnerabilities in OpenBSD, NetBSD 9.0 RC1 is available, Running FreeNAS on a DigitalOcean droplet, NomadBSD 1.3 is here, at e2k19 nobody can hear you scream, and more. Headlines Authentication vulnerabilities in OpenBSD (https://www.openwall.com/lists/oss-security/2019/12/04/5) We discovered an authentication-bypass vulnerability in OpenBSD's authentication system: this vulnerability is remotely exploitable in smtpd, ldapd, and radiusd, but its real-world impact should be studied on a case-by-case basis. For example, sshd is not exploitable thanks to its defense-in-depth mechanisms. From the manual page of login.conf: OpenBSD uses BSD Authentication, which is made up of a variety of authentication styles. The authentication styles currently provided are: passwd Request a password and check it against the password in the master.passwd file. See loginpasswd(8). skey Send a challenge and request a response, checking it with S/Key (tm) authentication. See loginskey(8). yubikey Authenticate using a Yubico YubiKey token. See loginyubikey(8). For any given style, the program /usr/libexec/auth/loginstyle is used to perform the authentication. The synopsis of this program is: /usr/libexec/auth/login_style [-v name=value] [-s service] username class This is the first piece of the puzzle: if an attacker specifies a username of the form "-option", they can influence the behavior of the authentication program in unexpected ways. login_passwd [-s service] [-v wheel=yes|no] [-v lastchance=yes|no] user [class] The service argument specifies which protocol to use with the invoking program. The allowed protocols are login, challenge, and response. (The challenge protocol is silently ignored but will report success as passwd-style authentication is not challenge-response based). This is the second piece of the puzzle: if an attacker specifies the username "-schallenge" (or "-schallenge:passwd" to force a passwd-style authentication), then the authentication is automatically successful and therefore bypassed. Case study: smtpd Case study: ldapd Case study: radiusd Case study: sshd Acknowledgments: We thank Theo de Raadt and the OpenBSD developers for their incredibly quick response: they published patches for these vulnerabilities less than 40 hours after our initial contact. We also thank MITRE's CVE Assignment Team. First release candidate for NetBSD 9.0 available! (https://blog.netbsd.org/tnf/entry/first_release_candidate_for_netbsd) Since the start of the release process four months ago a lot of improvements went into the branch - more than 500 pullups were processed! This includes usbnet (a common framework for usb ethernet drivers), aarch64 stability enhancements and lots of new hardware support, installer/sysinst fixes and changes to the NVMM (hardware virtualization) interface. We hope this will lead to the best NetBSD release ever (only to be topped by NetBSD 10 next year). Here are a few highlights of the new release: Support for Arm AArch64 (64-bit Armv8-A) machines, including "Arm ServerReady" compliant machines (SBBR+SBSA) Enhanced hardware support for Armv7-A Updated GPU drivers (e.g. support for Intel Kabylake) Enhanced virtualization support Support for hardware-accelerated virtualization (NVMM) Support for Performance Monitoring Counters Support for Kernel ASLR Support several kernel sanitizers (KLEAK, KASAN, KUBSAN) Support for userland sanitizers Audit of the network stack Many improvements in NPF Updated ZFS Reworked error handling and NCQ support in the SATA subsystem Support a common framework for USB Ethernet drivers (usbnet) More information on the RC can be found on the NetBSD 9 release page (https://www.netbsd.org/releases/formal-9/NetBSD-9.0.html) News Roundup Running FreeNAS on a Digitalocean droplet (https://www.shlomimarco.com/post/running-freenas-on-a-digitalocean-droplet) ZFS is awesome. FreeBSD even more so. FreeNAS is the battle-tested, enterprise-ready-yet-home-user-friendly software defined storage solution which is cooler then deep space, based on FreeBSD and makes heavy use of ZFS. This is what I (and soooooo many others) use for just about any storage-related task. I can go on and on and on about what makes it great, but if you're here, reading this, you probably know all that already and we can skip ahead. I've needed an offsite FreeNAS setup to replicate things to, to run some things, to do some stuff, basically, my privately-owned, tightly-controlled NAS appliance in the cloud, one I control from top to bottom and with support for whatever crazy thing I'm trying to do. Since I'm using DigitalOcean as my main VPS provider, it seemed logical to run FreeNAS there, however, you can't. While DO supports many many distos and pre-setup applications (e.g OpenVPN), FreeNAS isn't a supported feature, at least not in the traditional way :) Before we begin, here's the gist of what we're going to do: Base of a FreeBSD droplet, we'll re-image our boot block device with FreeNAS iso. We'll then install FreeNAS on the second block device. Once done we're going to do the ol' switcheroo: we're going to re-image our original boot block device using the now FreeNAS-installed second block device. Part 1: re-image our boot block device to boot FreeNAS install media. Part 2: Install FreeNAS on the second block-device Part 3: Re-image the boot block device using the FreeNAS-installed block device NomadBSD 1.3 is now available (https://nomadbsd.org/) From the release notes: The base system has been changed to FreeBSD 12.1-RELEASE-p1 Due to a deadlock problem, FreeBSD's unionfs has been replaced by unionfs-fuse The GPT layout has been changed to MBR. This prevents problems with Lenovo systems that refuse to boot from GPT if "lenovofix" is not set, and systems that hang on boot if "lenovofix" is set. Support for ZFS installations has been added to the NomadBSD installer. The rc-script for setting up the network interfaces has been fixed and improved. Support for setting the country code for the wlan device has been added. Auto configuration for running in VirtualBox has been added. A check for the default display has been added to the graphics configuration scripts. This fixes problems where users with Optimus have their NVIDIA card disabled, and use the integrated graphics chip instead. NVIDIA driver version 440 has been added. nomadbsd-dmconfig, a Qt tool for selecting the display manager theme, setting the default user and autologin has been added. nomadbsd-adduser, a Qt tool for added preconfigured user accounts to the system has been added. Martin Orszulik added Czech translations to the setup and installation wizard. The NomadBSD logo, designed by Ian Grindley, has been changed. Support for localized error messages has been added. Support for localizing the password prompts has been added. Some templates for starting other DEs have been added to ~/.xinitrc. The interfaces of nomadbsd-setup-gui and nomadbsd-install-gui have been improved. A script that helps users to configure a multihead systems has been added. The Xorg driver for newer Intel GPUs has been changed from "intel" to "modesetting". /proc has been added to /etc/fstab A D-Bus session issue has been fixed which prevented thunar from accessing samba shares. DSBBg which allows users to change and manage wallpapers has been added. The latest version of update_obmenu now supports auto-updating the Openbox menu. Manually updating the Openbox menu after packet (de)installation is therefore no longer needed. Support for multiple keyboard layouts has been added. www/palemoon has been removed. mail/thunderbird has been removed. audio/audacity has been added. deskutils/orage has been added. the password manager fpm2 has been replaced by KeePassXC mail/sylpheed has been replaced by mail/claws-mail multimedia/simplescreenrecorder has been added. DSBMC has been changed to DSBMC-Qt Many small improvements and bug fixes. At e2k19 nobody can hear you scream (https://undeadly.org/cgi?action=article;sid=20191204170908) After 2 years it was once again time to pack skis and snowshoes, put a satellite dish onto a sledge and hike through the snowy rockies to the Elk Lakes hut. I did not really have much of a plan what I wanted to work on but there were a few things I wanted to look into. One of them was rpki-client and the fact that it was so incredibly slow. Since Bob beck@ was around I started to ask him innocent X509 questions ... as if there are innocent X509 questions! Mainly about the abuse of the X509STORE in rpki-client. Pretty soon it was clear that rpki-client did it all wrong and most of the X509 verification had to be rewritten. Instead of only storing the root certificates in the store and passing the intermediate certs as a chain to the verification function rpki-client threw everything into it. The X509STORE is just not built for such an abuse and so it was no wonder that this was slow. Lucky me I pulled benno@ with me into this dark hole of libcrypto code. He managed to build up an initial diff to pass the chains as a STACKOF(X509) and together we managed to get it working. A big thanks goes to ingo@ who documented most of the functions we had to use. Have a look at STACKOF(3) and skpopfree(3) to understand why benno@ and I slowly turned crazy. Our next challenge was to only load the necessary certificate revocation list into the X509STORECTX. While doing those changes it became obvious that some of the data structures needed better lookup functions. Looking up certificates was done using a linear lookup and so we replaced the internal certificate and CRL tables with RB trees for fast lookups. deraadt@ also joined the rpki-client commit fest and changed the output code to use rename(2) so that files are replaced in an atomic operation. Thanks to this rpki-client can now be safely run from cron (there is an example in the default crontab). I did not plan to spend most of my week hacking on rpki-client but in the end I'm happy that I did and the result is fairly impressive. Working with libcrypto code and especially X509 was less than pleasant. Our screams of agony died away in the snowy rocky mountains and made Bob deep dive into UVM with a smile since he knew that benno@ and I had it worse. In case you wonder thanks to all changes at e2k19 rpki-client improved from over 20min run time to validate all VRPS to roughly 1min to do the same job. A factor 20 improvement! Thanks to Theo, Bob and Howie to make this possible. To all the cooks for the great food and to Xplornet for providing us with Internet at the hut. Beastie Bits FOSDEM 2020 BSD Devroom schedule (https://fosdem.org/2020/schedule/track/bsd/) Easy Minecraft Server on FreeBSD Howto (https://www.freebsdfoundation.org/freebsd/how-to-guides/easy-minecraft-server-on-freebsd/) stats(3) framework in the TCP stack (https://svnweb.freebsd.org/base?view=revision&revision=355304) 4017 days of uptime (https://twitter.com/EdwinKremer/status/1203071684535889921) sysget - A front-end for every package manager (https://github.com/emilengler/sysget) PlayOnBSD’s Cross-BSD Shopping Guide (https://www.playonbsd.com/shopping_guide/) Feedback/Questions Pat asks about the proper disk drive type for ZFS (http://dpaste.com/2FDN26X#wrap) Brad asks about a ZFS rosetta stone (http://dpaste.com/2X8PBMC#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag. Special Guest: Mariusz Zaborski.
Thu, 26 Dec 2019 05:00:00 -0800
329: Lucas’ Arts
In this episode, we interview Michael W. Lucas about his latest book projects, including the upcoming SNMP Mastery book. Interview - Michael Lucas Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag. Special Guest: Michael W Lucas.
Thu, 19 Dec 2019 05:00:00 -0800
328: EPYC Netflix Stack
LLDB Threading support now ready, Multiple IPSec VPN tunnels with FreeBSD, Netflix Optimized FreeBSD's Network Stack More Than Doubled AMD EPYC Performance, happy eyeballs with unwind(8), AWS got FreeBSD ARM 12, OpenSSH U2F/FIDO support, and more. Headlines LLDB Threading support now ready for mainline (https://blog.netbsd.org/tnf/entry/lldb_threading_support_now_ready) Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report. So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report. Multiple IPSec VPN tunnels with FreeBSD (https://blog.socruel.nu/text-only/how-to-multiple-ipsec-vpn-tunnels-on-freebsd.txt) The FreeBSD handbook describes an IPSec VPN tunnel between 2 FreeBSD hosts (see https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ipsec.html) But it is also possible to have multiple, 2 or more, IPSec VPN tunnels created and running on a FreeBSD host. How to implement and configure this is described below. The requirements is to have 3 locations (A, B and C) connected with IPSec VPN tunnels using FreeBSD (11.3-RELEASE). Each location has 1 IPSec VPN host running FreeBSD (VPN host A, B and C). VPN host A has 2 IPSec VPN tunnels: 1 to location B (VPN host B) and 1 to location C (VPN host C). News Roundup Netflix Optimized FreeBSD's Network Stack More Than Doubled AMD EPYC Performance (https://www.phoronix.com/scan.php?page=news_item&px=Netflix-NUMA-FreeBSD-Optimized) Drew Gallatin of Netflix presented at the recent EuroBSDcon 2019 conference in Norway on the company's network stack optimizations to FreeBSD. Netflix was working on being able to deliver 200Gb/s network performance for video streaming out of Intel Xeon and AMD EPYC servers, to which they are now at 190Gb/s+ and in the process that doubled the potential of EPYC Naples/Rome servers and also very hefty upgrades too for Intel. Netflix has long been known to be using FreeBSD in their data centers particularly where network performance is concerned. But in wanting to deliver 200Gb/s throughput from individual servers led them to making NUMA optimizations to the FreeBSD network stack. Allocating NUMA local memory for kernel TLS crypto buffers and for backing files sent via sentfile were among their optimizations. Changes to network connection handling and dealing with incoming connections to Nginx were also made. For those just wanting the end result, Netflix's NUMA optimizations to FreeBSD resulted in their Intel Xeon servers going from 105Gb/s to 191Gb/s while the NUMA fabric utilization dropped from 40% to 13%. unwind(8); "happy eyeballs" (https://marc.info/?l=openbsd-tech&m=157475113130337&w=2) In case you are wondering why happy eyeballs: It's a variation on this: https://en.wikipedia.org/wiki/Happy_Eyeballs unwind has a concept of a best nameserver type. It considers a configured DoT nameserver to be better than doing it's own recursive resolving. Recursive resolving is considered to be better than asking the dhcp provided nameservers. This diff sorts the nameserver types by quality, as above (validation, resolving, dead...), and as a tie breaker it adds the median of the round trip time of previous queries into the mix. One other interesting thing about this is that it gets us past captive portals without a check URL, that's why this diff is so huge, it rips out all the captive portal stuff (please apply with patch -E): 17 files changed, 385 insertions(+), 1683 deletions(-) Please test this. I'm particularly interested in reports from people who move between networks and need to get past captive portals. Amazon now has FreeBSD ARM 12 (https://aws.amazon.com/marketplace/pp/B081NF7BY7) Product Overview FreeBSD is an operating system used to power servers, desktops, and embedded systems. Derived from BSD, the version of UNIX developed at the University of California, Berkeley, FreeBSD has been continually developed by a large community for more than 30 years. FreeBSD's networking, security, storage, and monitoring features, including the pf firewall, the Capsicum and CloudABI capability frameworks, the ZFS filesystem, and the DTrace dynamic tracing framework, make FreeBSD the platform of choice for many of the busiest web sites and most pervasive embedded networking and storage systems. OpenSSH U2F/FIDO support in base (https://www.undeadly.org/cgi?action=article;sid=20191115064850) I just committed all the dependencies for OpenSSH security key (U2F) support to base and tweaked OpenSSH to use them directly. This means there will be no additional configuration hoops to jump through to use U2F/FIDO2 security keys. Hardware backed keys can be generated using "ssh-keygen -t ecdsa-sk" (or "ed25519-sk" if your token supports it). Many tokens require to be touched/tapped to confirm this step. You'll get a public/private keypair back as usual, except in this case, the private key file does not contain a highly-sensitive private key but instead holds a "key handle" that is used by the security key to derive the real private key at signing time. So, stealing a copy of the private key file without also stealing your security key (or access to it) should not give the attacker anything. Once you have generated a key, you can use it normally - i.e. add it to an agent, copy it to your destination's authorized_keys files (assuming they are running -current too), etc. At authentication time, you will be prompted to tap your security key to confirm the signature operation - this makes theft-of-access attacks against security keys more difficult too. Please test this thoroughly - it's a big change that we want to have stable before the next release. Beastie Bits DragonFly - git: virtio - Fix LUN scan issue w/ Google Cloud (http://lists.dragonflybsd.org/pipermail/commits/2019-November/719945.html) Really fast Markov chains in ~20 lines of sh, grep, cut and awk (https://0x0f0f0f.github.io/posts/2019/11/really-fast-markov-chains-in-~20-lines-of-sh-grep-cut-and-awk/) FreeBSD Journal Sept/Oct 2019 (https://www.freebsdfoundation.org/past-issues/security-3/) Michael Dexter is raising money for Bhyve development (https://twitter.com/michaeldexter/status/1201231729228308480) syscall call-from verification (https://marc.info/?l=openbsd-tech&m=157488907117170) FreeBSD Forums Howto Section (https://forums.freebsd.org/forums/howtos-and-faqs-moderated.39/) Feedback/Questions Jeroen - Feedback (http://dpaste.com/0PK1EG2#wrap) Savo - pfsense ports (http://dpaste.com/0PZ03B7#wrap) Tin - I want to learn C (http://dpaste.com/2GVNCYB#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 12 Dec 2019 04:00:00 -0800
327: ZFS Rename Repo
We read FreeBSD’s third quarterly status report, OpenBSD on Sparc64, ZoL repo move to OpenZFS, GEOM NOP, keeping NetBSD up-to-date, and more. Headlines FreeBSD third quarterly status report for 2019 (https://www.freebsd.org/news/status/report-2019-07-2019-09.html) This quarter the reports team has been more active than usual thanks to a better organization: calls for reports and reminders have been sent regularly, reports have been reviewed and merged quickly (I would like to thank debdrup@ in particular for his reviewing work). Efficiency could still be improved with the help of our community. In particular, the quarterly team has found that many reports have arrived in the last days before the deadline or even after. I would like to invite the community to follow the guidelines below that can help us sending out the reports sooner. Starting from next quarter, all quarterly status reports will be prepared the last month of the quarter itself, instead of the first month after the quarter's end. This means that deadlines for submitting reports will be the 1st of January, April, July and October. Next quarter will then be a short one, covering the months of November and December only and the report will probably be out in mid January. OpenBSD on Sparc64 (https://eerielinux.wordpress.com/2019/10/10/openbsd-on-sparc64-6-0-to-6-5/) OpenBSD, huh? Yes, I usually write about FreeBSD and that’s in fact what I tried installing on the machine first. But I ran into problems with it very early on (never even reached single user mode) and put it aside for later. Since I powered up the SunFire again last month, I needed an OS now and chose OpenBSD for the simple reason that I have it available. First I wanted to call this article simply “OpenBSD on SPARC” – but that would have been misleading since OpenBSD used to support 32-bit SPARC processors, too. The platform was just put to rest after the 5.9 release. Version 6.0 was the last release of OpenBSD that came on CD-ROM. When I bought it, I thought that I’d never use the SPARC CD. But here was the chance! While it is an obsolete release, it comes with the cryptographic signatures to verify the next release. So the plan is to start at 6.0 as I can trust the original CDs and then update to the latest release. This will also be an opportunity to recap on some of the things that changed over the various versions. News Roundup ZoL repo move to OpenZFS (https://zfsonlinux.topicbox.com/groups/zfs-discuss/T13eedc32607dab41/zol-repo-move-to-openzfs) Because it will contain the ZFS source code for both Linux and FreeBSD, we will rename the "ZFSonLinux" code repository to "OpenZFS". Specifically, the repo at http://github.com/ZFSonLinux/zfs will be moved to the OpenZFS organization, at http://github.com/OpenZFS/zfs. The next major release of ZFS for Linux and FreeBSD will be "OpenZFS 2.0", and is expected to ship in 2020. Mcclure111 Sun Thread (https://twitter.com/mcclure111/status/1196557401710837762) A long time ago— like 15 years ago— I worked at Sun Microsystems. The company was nearly dead at the time (it died a couple years later) because they didn't make anything that anyone wanted to buy anymore. So they had a lot of strange ideas about how they'd make their comeback. GEOM NOP (https://oshogbo.vexillium.org/blog/71/) Sometimes while testing file systems or applications you want to simulate some errors on the disk level. The first time I heard about this need was from Baptiste Daroussin during his presentation at AsiaBSDCon 2016. He mentioned how they had built a test lab with it. The same need was recently discussed during the PGCon 2019, to test a PostgreSQL instance. If you are FreeBSD user, I have great news for you: there is a GEOM provider which allows you to simulate a failing device. GNOP allows us to configure transparent providers from existing ones. The first interesting option of it is that we can slice the device into smaller pieces, thanks to the ‘offset option’ and ‘stripsesize’. This allows us to observe how the data on the disk is changing. Let’s assume that we want to observe the changes in the GPT table when the GPT flags are added or removed (for example the bootme flags which are described here). We can use dd every time and analyze it using absolute values from the disks. Keeping NetBSD up-to-date with pkg_comp 2.0 (https://jmmv.dev/2017/02/pkg_comp-2.0-tutorial-netbsd.html) This is a tutorial to guide you through the shiny new pkg_comp 2.0 on NetBSD. Goals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your NetBSD system up-to-date and secure. This tutorial is specifically targeted at NetBSD but should work on other platforms with some small changes. Expect, at the very least, a macOS-specific tutorial as soon as I create a pkg_comp standalone installer for that platform. Beastie Bits DragonFly - Radeon Improvements (http://lists.dragonflybsd.org/pipermail/commits/2019-November/720070.html) NomadBSD review (https://www.youtube.com/watch?v=7DglP7SbnlA&feature=share) Spongebob OpenBSD Security Comic (https://files.yukiisbo.red/openbsd_claim.png) Forth : The Early Years (https://colorforth.github.io/HOPL.html) LCM+L PDP-7 booting and running UNIX Version 0 (https://www.youtube.com/watch?v=pvaPaWyiuLA) Feedback/Questions Chris - Ctrl-T (http://dpaste.com/284E5BV) Improved Ctrl+t that shows kernel backtrace (https://asciinema.org/a/xfSpvPT61Cnd9iRgbfIjT6kYj) Brian - Migrating NexentaStore to FreeBSD/FreeNAS (http://dpaste.com/05GDK8H#wrap) Avery - How to get involved (http://dpaste.com/26KW801#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 05 Dec 2019 04:00:00 -0800
326: Certified BSD
LPI releases BSD Certification, openzfs trip report, Using FreeBSD with ports, LLDB threading support ready, Linux versus Open Source Unix, and more. Headlines Linux Professional Institute Releases BSD Specialist Certification - re BSD Certification Group (https://www.lpi.org/articles/linux-professional-institute-releases-bsd-specialist-certification) Linux Professional Institute extends its Open Technology certification track with the BSD Specialist Certification. Starting October 30, 2019, BSD Specialist exams will be globally available. The certification was developed in collaboration with the BSD Certification Group which merged with Linux Professional Institute in 2018. G. Matthew Rice, the Executive Director of Linux Professional Institute says that "the release of the BSD Specialist certification marks a major milestone for Linux Professional Institute. With this new credential, we are reaffirming our belief in the value of, and support for, all open source technologies. As much as possible, future credentials and educational programs will include coverage of BSD.” OpenZFS Trip Report (https://www.ixsystems.com/blog/openzfs-dev-summit-2019/) The seventh annual OpenZFS Developer Summit took place on November 4th and 5th in San Francisco and brought together a healthy mix of familiar faces and new community participants. Several folks from iXsystems took part in the talks, hacking, and socializing at this amazing annual event. The messages of the event can be summed up as Unification, Refinement, and Ecosystem Tooling. News Roundup Using FreeBSD with Ports (2/2): Tool-assisted updating (https://eerielinux.wordpress.com/2019/09/12/using-freebsd-with-ports-2-2-tool-assisted-updating/) Part 1 here: https://eerielinux.wordpress.com/2019/08/18/using-freebsd-with-ports-1-2-classic-way-with-tools/ In the previous post I explained why sometimes building your software from ports may make sense on FreeBSD. I also introduced the reader to the old-fashioned way of using tools to make working with ports a bit more convenient. In this follow-up post we’re going to take a closer look at portmaster and see how it especially makes updating from ports much, much easier. For people coming here without having read the previous article: What I describe here is not what every FreeBSD admin today should consider good practice (any more)! It can still be useful in special cases, but my main intention is to discuss this for building up the foundation for what you actually should do today. LLDB Threading support now ready for mainline (http://blog.netbsd.org/tnf/entry/lldb_threading_support_now_ready) Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report. So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report. Linux VS open source UNIX (https://www.adminbyaccident.com/politics/linux-vs-open-source-unix/) Beastie Bits Support for Realtek RTL8125 2.5Gb Ethernet controller (https://marc.info/?l=openbsd-tech&m=157380442230074&w=2) Computer Files Are Going Extinct (https://onezero.medium.com/the-death-of-the-computer-file-doc-43cb028c0506) FreeBSD kernel hacking (https://www.youtube.com/watch?v=4FUub_UtF3c) Modern BSD Computing for Fun on a VAX! Trying to use a VAX in today's world by Jeff Armstrong (https://youtu.be/e7cJ7v2lYdE) MidnightBSD 1.2 Released (https://www.justjournal.com/users/mbsd/entry/33779) Feedback/Questions Paulo - Zfs snapshots (http://dpaste.com/0WQRP43#wrap) Phillip - GCP (http://dpaste.com/075ZQE1#wrap) A Listener - Old episodes? (http://dpaste.com/3YJ4119#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 28 Nov 2019 04:00:00 -0800
325: Cracking Rainbows
FreeBSD 12.1 is here, A history of Unix before Berkeley, FreeBSD development setup, HardenedBSD 2019 Status Report, DNSSEC, compiling RainbowCrack on OpenBSD, and more. Headlines FreeBSD 12.1 (https://www.freebsd.org/releases/12.1R/announce.html) Some of the highlights: BearSSL has been imported to the base system. The clang, llvm, lld, lldb, compiler-rt utilities and libc++ have been updated to version 8.0.1. OpenSSL has been updated to version 1.1.1d. Several userland utility updates. For a complete list of new features and known problems, please see the online release notes and errata list, available at: https://www.FreeBSD.org/releases/12.1R/relnotes.html A History of UNIX before Berkeley: UNIX Evolution: 1975-1984. (http://www.darwinsys.com/history/hist.html) Nobody needs to be told that UNIX is popular today. In this article we will show you a little of where it was yesterday and over the past decade. And, without meaning in the least to minimise the incredible contributions of Ken Thompson and Dennis Ritchie, we will bring to light many of the others who worked on early versions, and try to show where some of the key ideas came from, and how they got into the UNIX of today. Our title says we are talking about UNIX evolution. Evolution means different things to different people. We use the term loosely, to describe the change over time among the many different UNIX variants in use both inside and outside Bell Labs. Ideas, code, and useful programs seem to have made their way back and forth - like mutant genes - among all the many UNIXes living in the phone company over the decade in question. Part One looks at some of the major components of the current UNIX system - the text formatting tools, the compilers and program development tools, and so on. Most of the work described in Part One took place at Research'', a part of Bell Laboratories (now AT&T Bell Laboratories, then as nowthe Labs''), and the ancestral home of UNIX. In planned (but not written) later parts, we would have looked at some of the myriad versions of UNIX - there are far more than one might suspect. This includes a look at Columbus and USG and at Berkeley Unix. You'll begin to get a glimpse inside the history of the major streams of development of the system during that time. News Roundup My FreeBSD Development Setup (https://adventurist.me/posts/00296) I do my FreeBSD development using git, tmux, vim and cscope. I keep a FreeBSD fork on my github, I have forked https://github.com/freebsd/freebsd to https://github.com/adventureloop/freebsd OPNsense 19.7.6 released (https://opnsense.org/opnsense-19-7-6-released/) As we are experiencing the Suricata community first hand in Amsterdam we thought to release this version a bit earlier than planned. Included is the latest Suricata 5.0.0 release in the development version. That means later this November we will releasing version 5 to the production version as we finish up tweaking the integration and maybe pick up 5.0.1 as it becomes available. LDAP TLS connectivity is now integrated into the system trust store, which ensures that all required root and intermediate certificates will be seen by the connection setup when they have been added to the authorities section. The same is true for trusting self-signed certificates. On top of this, IPsec now supports public key authentication as contributed by Pascal Mathis. HardenedBSD November 2019 Status Report. (https://hardenedbsd.org/article/shawn-webb/2019-11-09/hardenedbsd-status-report) We at HardenedBSD have a lot of news to share. On 05 Nov 2019, Oliver Pinter resigned amicably from the project. All of us at HardenedBSD owe Oliver our gratitude and appreciation. This humble project, named by Oliver, was born out of his thesis work and the collaboration with Shawn Webb. Oliver created the HardenedBSD repo on GitHub in April 2013. The HardenedBSD Foundation was formed five years later to carry on this great work. DNSSEC enabled in default unbound(8) configuration. (https://undeadly.org/cgi?action=article;sid=20191110123908) DNSSEC validation has been enabled in the default unbound.conf(5) in -current. The relevant commits were from Job Snijders (job@) How to Install Shopware with NGINX and Let's Encrypt on FreeBSD 12 (https://www.howtoforge.com/how-to-install-shopware-with-nginx-and-lets-encrypt-on-freebsd-12/) Shopware is the next generation of open source e-commerce software. Based on bleeding edge technologies like Symfony 3, Doctrine2 and Zend Framework Shopware comes as the perfect platform for your next e-commerce project. This tutorial will walk you through the Shopware Community Edition (CE) installation on FreeBSD 12 system by using NGINX as a web server. Requirements Make sure your system meets the following minimum requirements: + Linux-based operating system with NGINX or Apache 2.x (with mod_rewrite) web server installed. + PHP 5.6.4 or higher with ctype, gd, curl, dom, hash, iconv, zip, json, mbstring, openssl, session, simplexml, xml, zlib, fileinfo, and pdo/mysql extensions. PHP 7.1 or above is strongly recommended. + MySQL 5.5.0 or higher. + Possibility to set up cron jobs. + Minimum 4 GB available hard disk space. + IonCube Loader version 5.0.0 or higher (optional). How to Compile RainbowCrack on OpenBSD (https://cromwell-intl.com/open-source/compiling-rainbowcrack-on-openbsd.html) Project RainbowCrack was originally Zhu Shuanglei's implementation, it's not clear to me if the project is still just his or if it's even been maintained for a while. His page seems to have been last updated in August 2007. The Project RainbowCrack web page now has just binaries for Windows XP and Linux, both 32-bit and 64-bit versions. Earlier versions were available as source code. The version 1.2 source code does not compile on OpenBSD, and in my experience it doesn't compile on Linux, either. It seems to date from 2004 at the earliest, and I think it makes some version-2.4 assumptions about Linux kernel headers. You might also look at ophcrack, a more modern tool, although it seems to be focused on cracking Windows XP/Vista/7/8/10 password hashes Feedback/Questions Reese - Amature radio info (http://dpaste.com/2RDG9K4#wrap) Chris - VPN (http://dpaste.com/2K4T2FQ#wrap) Malcolm - NAT (http://dpaste.com/138NEMA) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 21 Nov 2019 04:00:00 -0800
324: Emergency Space Mode
Migrating drives and zpool between hosts, OpenBSD in 2019, Dragonfly’s new zlib and dhcpcd, Batch renaming images and resolution with awk, a rant on the X11 ICCCM selection system, hammer 2 emergency space mode, and more. Headlines Migrating drives and the zpool from one host to another. (https://dan.langille.org/2019/10/26/migrating-drives-and-the-zpool-from-one-host-to-another/) Today is the day. Today I move a zpool from an R710 into an R720. The goal: all services on that zpool start running on the new host. Fortunately, that zpool is dedicated to jails, more or less. I have done some planning about this, including moving a poudriere on the R710 into a jail. Now it is almost noon on Saturday, I am sitting in the basement (just outside the server room), and I’m typing this up. In this post: FreeBSD 12.0 Dell R710 (r710-01) Dell R720 (r720-01) drive caddies from eBay and now I know the difference between SATA and SATAu PLEASE READ THIS first: Migrating ZFS Storage Pools (https://docs.oracle.com/cd/E19253-01/819-5461/gbchy/index.html) OpenBSD in 2019 (https://blog.habets.se/2019/10/OpenBSD-in-2019.html) I’ve used OpenBSD on and off since 2.1. More back then than in the last 10 years or so though, so I thought I’d try it again. What triggered this was me finding a silly bug in GNU cpio that has existed with a “FIXME” comment since at least 1994. I checked OpenBSD to see if it had a related bug, but as expected no it was just fine. I don’t quite remember why I stopped using OpenBSD for servers, but I do remember filesystem corruption on “unexpected power disconnections” (even with softdep turned on), which I’ve never really seen on Linux. That and that fewer things “just worked” than with Linux, which matters more when I installed more random things than I do now. I’ve become a lot more minimalist. Probably due to less spare time. Life is better when you don’t run things like PHP (not that OpenBSD doesn’t support PHP, just an example) or your own email server with various antispam tooling, and other things. This is all experience from running OpenBSD on a server. On my next laptop I intend to try running OpenBSD on the dektop, and will see if that more ad-hoc environment works well. E.g. will gnuradio work? Lack of other-OS VM support may be a problem. Verdict Ouch, that’s a long list of bad stuff. Still, I like it. I’ll continue to run it, and will make sure my stuff continues working on OpenBSD. And maybe in a year I’ll have a review of OpenBSD on a laptop. News Roundup New zlib, new dhcpcd (https://www.dragonflydigest.com/2019/10/29/23683.html) zlib and dhcpcd are both updated in DragonFly… but my quick perusal of the commits makes it sound like bugfix only; no usage changes needed. DHCPCD Commit: http://lists.dragonflybsd.org/pipermail/commits/2019-October/719768.html ZLIB Commit: http://lists.dragonflybsd.org/pipermail/commits/2019-October/719772.html Batch renaming images, including image resolution, with awk (https://victoria.dev/verbose/batch-renaming-images-including-image-resolution-with-awk/) The most recent item on my list of “Geeky things I did that made me feel pretty awesome” is an hour’s adventure that culminated in this code: $ file IMG* | awk 'BEGIN{a=0} {print substr($1, 1, length($1)-5),a++"_"substr($8,1, length($8)-1)}' | while read fn fr; do echo $(rename -v "s/$fn/img_$fr/g" *); done IMG_20170808_172653_425.jpg renamed as img_0_4032x3024.jpg IMG_20170808_173020_267.jpg renamed as img_1_3024x3506.jpg IMG_20170808_173130_616.jpg renamed as img_2_3024x3779.jpg IMG_20170808_173221_425.jpg renamed as img_3_3024x3780.jpg IMG_20170808_173417_059.jpg renamed as img_4_2956x2980.jpg IMG_20170808_173450_971.jpg renamed as img_5_3024x3024.jpg IMG_20170808_173536_034.jpg renamed as img_6_4032x3024.jpg IMG_20170808_173602_732.jpg renamed as img_7_1617x1617.jpg IMG_20170808_173645_339.jpg renamed as img_8_3024x3780.jpg IMG_20170909_170146_585.jpg renamed as img_9_3036x3036.jpg IMG_20170911_211522_543.jpg renamed as img_10_3036x3036.jpg IMG_20170913_071608_288.jpg renamed as img_11_2760x2760.jpg IMG_20170913_073205_522.jpg renamed as img_12_2738x2738.jpg // ... etc etc The last item on the aforementioned list is “TODO: come up with a shorter title for this list.” I hate the X11 ICCCM selection system, and you should too - A Rant (http://www.call-with-current-continuation.org/rants/icccm.txt) d00d, that document is devilspawn. I've recently spent my nights in pain implementing the selection mechanism. WHY OH WHY OH WHY? why me? why did I choose to do this? and what sick evil twisted mind wrote this damn spec? I don't know why I'm working with it, I just wanted to make a useful program. I didn't know what I was getting myself in to. Nobody knows until they try it. And once you start, you're unable to stop. You can't stop, if you stop then you haven't completed it to spec. You can't fail on this, it's just a few pages of text, how can that be so hard? So what if they use Atoms for everything. So what if there's no explicit correlation between the target type of a SelectionNotify event and the type of the property it indicates? So what if the distinction is ambiguous? So what if the document is littered with such atrocities? It's not the spec's fault, the spec is authoritative. It's obviously YOUR (the implementor's) fault for misunderstanding it. If you didn't misunderstand it, you wouldn't be here complaining about it would you? HAMMER2 emergency space mode (https://www.dragonflydigest.com/2019/10/22/23652.html) As anyone who has been running HAMMER1 or HAMMER2 has noticed, snapshots and copy on write and infinite history can eat a lot of disk space, even if the actual file volume isn’t changing much. There’s now an ‘emergency mode‘ for HAMMER2, where disk operations can happen even if there isn’t space for the normal history activity. It’s dangerous, in that the normal protections against data loss if power is cut go away, and snapshots created while in this mode will be mangled. So definitely don’t leave it on! Beastie Bits The BastilleBSD community has started work on over 100 automation templates (https://twitter.com/BastilleBSD/status/1186659762458501120) PAM perturbed (https://www.dragonflydigest.com/2019/10/23/23654.html) OpenBSD T-Shirts now available (https://teespring.com/stores/openbsd) FastoCloud (Opensource Media Service) now available on FreeBSD (https://old.reddit.com/r/freebsd/comments/dlyqtq/fastocloud_opensource_media_service_now_available/) Unix: A History and a Memoir by Brian Kernighan now available (https://www.cs.princeton.edu/~bwk/) OpenBSD Moonlight game streaming client from a Windows + Nvidia PC (https://www.reddit.com/r/openbsd_gaming/comments/d6xboo/openbsd_moonlight_game_streaming_client_from_a/) *** Feedback/Questions Tim - Release Notes for Lumina 1.5 (http://dpaste.com/38DNSXT#wrap) Answer Here (http://dpaste.com/3QJX8G3#wrap) Brad - vBSDcon Trip Report (http://dpaste.com/316MGVX#wrap) Jacob - Using terminfo on FreeBSD (http://dpaste.com/131N05J#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 14 Nov 2019 04:00:00 -0800
323: OSI Burrito Guy
The earliest Unix code, how to replace fail2ban with blacklistd, OpenBSD crossed 400k commits, how to install Bolt CMS on FreeBSD, optimized hammer2, appeasing the OSI 7-layer burrito guys, and more. Headlines The Earliest Unix Code: An Anniversary Source Code Release (https://computerhistory.org/blog/the-earliest-unix-code-an-anniversary-source-code-release/) What is it that runs the servers that hold our online world, be it the web or the cloud? What enables the mobile apps that are at the center of increasingly on-demand lives in the developed world and of mobile banking and messaging in the developing world? The answer is the operating system Unix and its many descendants: Linux, Android, BSD Unix, MacOS, iOS—the list goes on and on. Want to glimpse the Unix in your Mac? Open a Terminal window and enter “man roff” to view the Unix manual entry for an early text formatting program that lives within your operating system. 2019 marks the 50th anniversary of the start of Unix. In the summer of 1969, that same summer that saw humankind’s first steps on the surface of the Moon, computer scientists at the Bell Telephone Laboratories—most centrally Ken Thompson and Dennis Ritchie—began the construction of a new operating system, using a then-aging DEC PDP-7 computer at the labs. This man sent the first online message 50 years ago (https://www.cbc.ca/radio/thecurrent/the-current-for-oct-29-2019-1.5339212/this-man-sent-the-first-online-message-50-years-ago-he-s-since-seen-the-web-s-dark-side-emerge-1.5339244) As many of you have heard in the past, the first online message ever sent between two computers was "lo", just over 50 years ago, on Oct. 29, 1969. It was supposed to say "log," but the computer sending the message — based at UCLA — crashed before the letter "g" was typed. A computer at Stanford 560 kilometres away was supposed to fill in the remaining characters "in," as in "log in." The CBC Radio show, “The Current” has a half-hour interview with the man who sent that message, Leonard Kleinrock, distinguished professor of computer science at UCLA "The idea of the network was you could sit at one computer, log on through the network to a remote computer and use its services there," 50 years later, the internet has become so ubiquitous that it has almost been rendered invisible. There's hardly an aspect in our daily lives that hasn't been touched and transformed by it. Q: Take us back to that day 50 years ago. Did you have the sense that this was going to be something you'd be talking about a half a century later? A: Well, yes and no. Four months before that message was sent, there was a press release that came out of UCLA in which it quotes me as describing what my vision for this network would become. Basically what it said is that this network would be always on, always available. Anybody with any device could get on at anytime from any location, and it would be invisible. Well, what I missed ... was that this is going to become a social network. People talking to people. Not computers talking to computers, but [the] human element. Q: Can you briefly explain what you were working on in that lab? Why were you trying to get computers to actually talk to one another? A: As an MIT graduate student, years before, I recognized I was surrounded by computers and I realized there was no effective [or efficient] way for them to communicate. I did my dissertation, my research, on establishing a mathematical theory of how these networks would work. But there was no such network existing. AT&T said it won't work and, even if it does, we want nothing to do with it. So I had to wait around for years until the Advanced Research Projects Agency within the Department of Defence decided they needed a network to connect together the computer scientists they were supervising and supporting. Q: For all the promise of the internet, it has also developed some dark sides that I'm guessing pioneers like yourselves never anticipated. A: We did not. I knew everybody on the internet at that time, and they were all well-behaved and they all believed in an open, shared free network. So we did not put in any security controls. When the first spam email occurred, we began to see the dark side emerge as this network reached nefarious people sitting in basements with a high-speed connection, reaching out to millions of people instantaneously, at no cost in time or money, anonymously until all sorts of unpleasant events occurred, which we called the dark side. But in those early days, I considered the network to be going through its teenage years. Hacking to spam, annoying kinds of effects. I thought that one day this network would mature and grow up. Well, in fact, it took a turn for the worse when nation states, organized crime and extremists came in and began to abuse the network in severe ways. Q: Is there any part of you that regrets giving birth to this? A: Absolutely not. The greater good is much more important. News Roundup How to use blacklistd(8) with NPF as a fail2ban replacement (https://www.unitedbsd.com/d/63-how-to-use-blacklistd8-with-npf-as-a-fail2ban-replacement) blacklistd(8) provides an API that can be used by network daemons to communicate with a packet filter via a daemon to enforce opening and closing ports dynamically based on policy. The interface to the packet filter is in /libexec/blacklistd-helper (this is currently designed for npf) and the configuration file (inspired from inetd.conf) is in etc/blacklistd.conf Now, blacklistd(8) will require bpfjit(4) (Just-In-Time compiler for Berkeley Packet Filter) in order to properly work, in addition to, naturally, npf(7) as frontend and syslogd(8), as a backend to print diagnostic messages. Also remember npf shall rely on the npflog* virtual network interface to provide logging for tcpdump() to use. Unfortunately (dont' ask me why ??) in 8.1 all the required kernel components are still not compiled by default in the GENERIC kernel (though they are in HEAD), and are rather provided as modules. Enabling NPF and blacklistd services would normally result in them being automatically loaded as root, but predictably on securelevel=1 this is not going to happen. FreeBSD’s handbook chapter on blacklistd (https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/firewalls-blacklistd.html) OpenBSD crossed 400,000 commits (https://marc.info/?l=openbsd-tech&m=157059352620659&w=2) Sometime in the last week OpenBSD crossed 400,000 commits (*) upon all our repositories since starting at 1995/10/18 08:37:01 Canada/Mountain. That's a lot of commits by a lot of amazing people. (*) by one measure. Since the repository is so large and old, there are a variety of quirks including ChangeLog missing entries and branches not convertible to other repo forms, so measuring is hard. If you think you've got a great way of measuring, don't be so sure of yourself -- you may have overcounted or undercounted. Subject to the notes Theo made about under and over counting, FreeBSD should hit 1 million commits (base + ports + docs) some time in 2020 NetBSD + pkgsrc are approaching 600,000, but of course pkgsrc covers other operating systems too How to Install Bolt CMS with Nginx and Let's Encrypt on FreeBSD 12 (https://www.howtoforge.com/how-to-install-bolt-cms-nginx-ssl-on-freebsd-12/) Bolt is a sophisticated, lightweight and simple CMS built with PHP. It is released under the open-source MIT-license and source code is hosted as a public repository on Github. A bolt is a tool for Content Management, which strives to be as simple and straightforward as possible. It is quick to set up, easy to configure, uses elegant templates. Bolt is created using modern open-source libraries and is best suited to build sites in HTML5 with modern markup. In this tutorial, we will go through the Bolt CMS installation on FreeBSD 12 system by using Nginx as a web server, MySQL as a database server, and optionally you can secure the transport layer by using acme.sh client and Let's Encrypt certificate authority to add SSL support. Requirements The system requirements for Bolt are modest, and it should run on any fairly modern web server: PHP version 5.5.9 or higher with the following common PHP extensions: pdo, mysqlnd, pgsql, openssl, curl, gd, intl, json, mbstring, opcache, posix, xml, fileinfo, exif, zip. Access to SQLite (which comes bundled with PHP), or MySQL or PostgreSQL. Apache with mod_rewrite enabled (.htaccess files) or Nginx (virtual host configuration covered below). A minimum of 32MB of memory allocated to PHP. hammer2 - Optimize hammer2 support threads and dispatch (http://lists.dragonflybsd.org/pipermail/commits/2019-September/719632.html) Refactor the XOP groups in order to be able to queue strategy calls, whenever possible, to the same CPU as the issuer. This optimizes several cases and reduces unnecessary IPI traffic between cores. The next best thing to do would be to not queue certain XOPs to an H2 support thread at all, but I would like to keep the threads intact for later clustering work. The best scaling case for this is when one has a large number of user threads doing I/O. One instance of a single-threaded program on an otherwise idle machine might see a slightly reduction in performance but at the same time we completely avoid unnecessarily spamming all cores in the system on the behalf of a single program, so overhead is also significantly lower. This will tend to increase the number of H2 support threads since we need a certain degree of multiplication for domain separation. This should significantly increase I/O performance for multi-threaded workloads. You know, we might as well just run every network service over HTTPS/2 and build another six layers on top of that to appease the OSI 7-layer burrito guys (http://boston.conman.org/2019/10/17.1) I've seen the writing on the wall, and while for now you can configure Firefox not to use DoH, I'm not confident enough to think it will remain that way. To that end, I've finally set up my own DoH server for use at Chez Boca. It only involved setting up my own CA to generate the appropriate certificates, install my CA certificate into Firefox, configure Apache to run over HTTP/2 (THANK YOU SO VERY XXXXX­XX MUCH GOOGLE FOR SHOVING THIS HTTP/2 XXXXX­XXX DOWN OUR THROATS!—no, I'm not bitter) and write a 150 line script that just queries my own local DNS, because, you know, it's more XXXXX­XX secure or some XXXXX­XXX reason like that. Sigh. Beastie Bits An Oral History of Unix (https://www.princeton.edu/~hos/Mahoney/unixhistory) NUMA Siloing in the FreeBSD Network Stack [pdf] (https://people.freebsd.org/~gallatin/talks/euro2019.pdf) EuroBSDCon 2019 videos available (https://www.youtube.com/playlist?list=PLskKNopggjc6NssLc8GEGSiFYJLYdlTQx) Barbie knows best (https://twitter.com/eksffa/status/1188638425567682560) For the #OpenBSD #e2k19 attendees. I did a pre visit today. (https://twitter.com/bob_beck/status/1188226661684301824) Drawer Find (https://twitter.com/pasha_sh/status/1187877745499561985) Slides - Removing ROP Gadgets from OpenBSD - AsiaBSDCon 2019 (https://www.openbsd.org/papers/asiabsdcon2019-rop-slides.pdf) Feedback/Questions Bostjan - Open source doesn't mean secure (http://dpaste.com/1M5MVCX#wrap) Malcolm - Allan is Correct. (http://dpaste.com/2RFNR94) Michael - FreeNAS inside a Jail (http://dpaste.com/28YW3BB#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 07 Nov 2019 04:00:00 -0800
322: Happy Birthday, Unix
Unix is 50, Hunting down Ken's PDP-7, OpenBSD and OPNSense have new releases, Clarification on what GhostBSD is, sshuttle - VPN over SSH, and more. Headlines Unix is 50 (https://www.bell-labs.com/unix50/) In the summer of 1969 computer scientists Ken Thompson and Dennis Ritchie created the first implementation of Unix with the goal of designing an elegant and economical operating system for a little-used PDP-7 minicomputer at Bell Labs. That modest project, however, would have a far-reaching legacy. Unix made large-scale networking of diverse computing systems — and the Internet — practical. The Unix team went on to develop the C language, which brought an unprecedented combination of efficiency and expressiveness to programming. Both made computing more "portable". Today, Linux, the most popular descendent of Unix, powers the vast majority of servers, and elements of Unix and Linux are found in most mobile devices. Meanwhile C++ remains one of the most widely used programming languages today. Unix may be a half-century old but its influence is only growing. Hunting down Ken's PDP-7: video footage found (https://bsdimp.blogspot.com/2019/10/video-footage-of-first-pdp-7-to-run-unix.html) In my prior blog post, I traced Ken's scrounged PDP-7 to SN 34. In this post I'll show that we have actual video footage of that PDP-7 due to an old film from Bell Labs. this gives us almost a minute of footage of the PDP-7 Ken later used to create Unix. News Roundup OpenBSD 6.6 Released (https://openbsd.org/66.html) Announce: https://marc.info/?l=openbsd-tech&m=157132024225971&w=2 Upgrade Guide: https://openbsd.org/faq/upgrade66.html Changelog: https://openbsd.org/plus66.html OPNsense 19.7.5 released (https://opnsense.org/opnsense-19-7-5-released/) Hello friends and followers, Lots of plugin and ports updates this time with a few minor improvements in all core areas. Behind the scenes we are starting to migrate the base system to version 12.1 which is supposed to hit the next 20.1 release. Stay tuned for more infos in the next month or so. Here are the full patch notes: + system: show all swap partitions in system information widget + system: flatten services_get() in preparation for removal + system: pin Syslog-ng version to specific package name + system: fix LDAP/StartTLS with user import page + system: fix a PHP warning on authentication server page + system: replace most subprocess.call use + interfaces: fix devd handling of carp devices (contributed by stumbaumr) + firewall: improve firewall rules inline toggles + firewall: only allow TCP flags on TCP protocol + firewall: simplify help text for direction setting + firewall: make protocol log summary case insensitive + reporting: ignore malformed flow records + captive portal: fix type mismatch for timeout read + dhcp: add note for static lease limitation with lease registration (contributed by Northguy) + ipsec: add margintime and rekeyfuzz options + ipsec: clear $dpdline correctly if not set + ui: fix tokenizer reorder on multiple saves + plugins: os-acme-client 1.26[1] + plugins: os-bind will reload bind on record change (contributed by blablup) + plugins: os-etpro-telemetry minor subprocess.call replacement + plugins: os-freeradius 1.9.4[2] + plugins: os-frr 1.12[3] + plugins: os-haproxy 2.19[4] + plugins: os-mailtrail 1.2[5] + plugins: os-postfix 1.11[6] + plugins: os-rspamd 1.8[7] + plugins: os-sunnyvalley LibreSSL support (contributed by Sunny Valley Networks) + plugins: os-telegraf 1.7.6[8] + plugins: os-theme-cicada 1.21 (contributed by Team Rebellion) + plugins: os-theme-tukan 1.21 (contributed by Team Rebellion) + plugins: os-tinc minor subprocess.call replacement + plugins: os-tor 1.8 adds dormant mode disable option (contributed by Fabian Franz) + plugins: os-virtualbox 1.0 (contributed by andrewhotlab) Dealing with the misunderstandings of what is GhostBSD (http://ghostbsd.org/node/194) Since the release of 19.09, I have seen a lot of misunderstandings on what is GhostBSD and the future of GhostBSD. GhostBSD is based on TrueOS with FreeBSD 12 STABLE with our twist to it. We are still continuing to use TrueOS for OpenRC, and the new package's system for the base system that is built from ports. GhostBSD is becoming a slow-moving rolling release base on the latest TrueOS with FreeBSD 12 STABLE. When FreeBSD 13 STABLE gets released, GhostBSD will be upgraded to TrueOS with FreeBSD 13 STABLE. Our official desktop is MATE, which means that the leading developer of GhostBSD does not officially support XFCE. Community releases are maintained by the community and for the community. GhostBSD project will provide help to build and to host the community release. If anyone wants to have a particular desktop supported, it is up to the community. Sure I will help where I can, answer questions and guide new community members that contribute to community release. There is some effort going on for Plasma5 desktop. If anyone is interested in helping with XFCE and Plasma5 or in creating another community release, you are well come to contribute. Also, Contribution to the GhostBSD base system, to ports and new ports, and in house software are welcome. We are mostly active on Telegram https://t.me/ghostbsd, but you can also reach us on the forum. SHUTTLE – VPN over SSH | VPN Alternative (https://www.terminalbytes.com/sshuttle-vpn-over-ssh-vpn-alternative/) Looking for a lightweight VPN client, but are not ready to spend a monthly recurring amount on a VPN? VPNs can be expensive depending upon the quality of service and amount of privacy you want. A good VPN plan can easily set you back by 10$ a month and even that doesn’t guarantee your privacy. There is no way to be sure whether the VPN is storing your confidential information and traffic logs or not. sshuttle is the answer to your problem it provides VPN over ssh and in this article we’re going to explore this cheap yet powerful alternative to the expensive VPNs. By using open source tools you can control your own privacy. VPN over SSH – sshuttle sshuttle is an awesome program that allows you to create a VPN connection from your local machine to any remote server that you have ssh access on. The tunnel established over the ssh connection can then be used to route all your traffic from client machine through the remote machine including all the dns traffic. In the bare bones sshuttle is just a proxy server which runs on the client machine and forwards all the traffic to a ssh tunnel. Since its open source it holds quite a lot of major advantages over traditional VPN. OpenSSH 8.1 Released (http://www.openssh.com/txt/release-8.1) Security ssh(1), sshd(8), ssh-add(1), ssh-keygen(1): an exploitable integer overflow bug was found in the private key parsing code for the XMSS key type. This key type is still experimental and support for it is not compiled by default. No user-facing autoconf option exists in portable OpenSSH to enable it. This bug was found by Adam Zabrocki and reported via SecuriTeam's SSD program. ssh(1), sshd(8), ssh-agent(1): add protection for private keys at rest in RAM against speculation and memory side-channel attacks like Spectre, Meltdown and Rambleed. This release encrypts private keys when they are not in use with a symmetric key that is derived from a relatively large "prekey" consisting of random data (currently 16KB). This release includes a number of changes that may affect existing configurations: ssh-keygen(1): when acting as a CA and signing certificates with an RSA key, default to using the rsa-sha2-512 signature algorithm. Certificates signed by RSA keys will therefore be incompatible with OpenSSH versions prior to 7.2 unless the default is overridden (using "ssh-keygen -t ssh-rsa -s ..."). New Features ssh(1): Allow %n to be expanded in ProxyCommand strings ssh(1), sshd(8): Allow prepending a list of algorithms to the default set by starting the list with the '^' character, E.g. "HostKeyAlgorithms ^ssh-ed25519" ssh-keygen(1): add an experimental lightweight signature and verification ability. Signatures may be made using regular ssh keys held on disk or stored in a ssh-agent and verified against an authorized_keys-like list of allowed keys. Signatures embed a namespace that prevents confusion and attacks between different usage domains (e.g. files vs email). ssh-keygen(1): print key comment when extracting public key from a private key. ssh-keygen(1): accept the verbose flag when searching for host keys in known hosts (i.e. "ssh-keygen -vF host") to print the matching host's random-art signature too. All: support PKCS8 as an optional format for storage of private keys to disk. The OpenSSH native key format remains the default, but PKCS8 is a superior format to PEM if interoperability with non-OpenSSH software is required, as it may use a less insecure key derivation function than PEM's. Beastie Bits Say goodbye to the 32 CPU limit in NetBSD/aarch64 (https://twitter.com/jmcwhatever/status/1185584719183962112) vBSDcon 2019 videos (https://www.youtube.com/channel/UCvcdrOSlYOSzOzLjv_n1_GQ/videos) Browse the web in the terminal - W3M (https://www.youtube.com/watch?v=3Hfda0Tjqsg&feature=youtu.be) NetBSD 9 and GSoC (http://netbsd.org/~kamil/GSoC2019.html#slide1) BSDCan 2019 Videos (https://www.youtube.com/playlist?list=PLeF8ZihVdpFegPoAKppaDSoYmsBvpnSZv) NYC*BUG Install Fest: Nov 6th 18:45 @ Suspenders (https://www.nycbug.org/index?action=view&id=10673) FreeBSD Miniconf at linux.conf.au 2020 Call for Sessions Now Open (https://www.freebsdfoundation.org/blog/freebsd-miniconf-at-linux-conf-au-2020-call-for-sessions-now-open/) FOSDEM 2020 - BSD Devroom Call for Participation (https://people.freebsd.org/~rodrigo/fosdem20/) University of Cambridge looking for Research Assistants/Associates (https://twitter.com/ed_maste/status/1184865668317007874) Feedback/Questions Trenton - Beeping Thinkpad (http://dpaste.com/0ZEXNM6#wrap) Alex - Per user ZFS Datasets (http://dpaste.com/1K31A65#wrap) Allan’s old patch from 2015 (https://reviews.freebsd.org/D2272) Javier - FBSD 12.0 + ZFS + encryption (http://dpaste.com/1XX4NNA#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 31 Oct 2019 04:00:00 -0700
321: The Robot OS
An interview with Trenton Schulz about his early days with FreeBSD, Robot OS, Qt, and more. Interview - Trenton Schulz - freenas@norwegianrockcat.com (mailto:freenas@norwegianrockcat.com) Robot OS on FreeBSD BR: Welcome to the show. Can you tell us a little bit about yourself and how you got started with BSD? AJ: You were working for Trolltech (creators of Qt). Was FreeBSD used there and how? BR: Can you tell us more about the work you are doing with Robot OS on FreeBSD? AJ: Was EuroBSDcon your first BSD conference? How did you like it? BR: Do you have some tips or advice on how to get started with the BSDs? AJ: Is there anything else you’d like to tell us before we let you go? Beastie Bits FreeBSD Miniconf at linux.conf.au 2020 Call for Sessions Now Open (https://www.freebsdfoundation.org/blog/freebsd-miniconf-at-linux-conf-au-2020-call-for-sessions-now-open/) Portland BSD Pizza Night: Oct 24th, 19:00 @ Rudy’s Gourmet Pizza (http://calagator.org/events/1250476319) NYC*BUG Install Fest: Nov 6th 18:45 @ Suspenders (https://www.nycbug.org/index?action=view&id=10673) FOSDEM 2020 - BSD Devroom Call for Participation (https://people.freebsd.org/~rodrigo/fosdem20/) University of Cambridge looking for Research Assistants/Associates (https://twitter.com/ed_maste/status/1184865668317007874) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag. Special Guest: Trenton Shulz.
Wed, 23 Oct 2019 20:00:00 -0700
320: Codebase: Neck Deep
Headlines FreeBSD and custom firmware on the Google Pixelbook (https://unrelenting.technology/articles/FreeBSD-and-custom-firmware-on-the-Google-Pixelbook) FreeBSD and custom firmware on the Google Pixelbook Back in 2015, I jumped on the ThinkPad bandwagon by getting an X240 to run FreeBSD on. Unlike most people in the ThinkPad crowd, I actually liked the clickpad and didn\u2019t use the trackpoint much. But this summer I\u2019ve decided that it was time for something newer. I wanted something.. lighter and thinner (ha, turns out this is actually important, I got tired of carrying a T H I C C laptop - Apple was right all along); with a 3:2 display (why is Lenovo making these Serious Work\u2122 laptops 16:9 in the first place?? 16:9 is awful in below-13-inch sizes especially); with a HiDPI display (and ideally with a good size for exact 2x scaling instead of fractional); with USB-C ports; without a dGPU, especially without an NVIDIA GPU; assembled with screws and not glue (I don\u2019t necessarily need expansion and stuff in a laptop all that much, but being able to replace the battery without dealing with a glued chassis is good); supported by FreeBSD of course (\u201csome development required\u201d is okay but I\u2019m not going to write big drivers); how about something with open source firmware, that would be fun. I was considering a ThinkPad X1 Carbon from an old generation - the one from the same year as the X230 is corebootable, so that\u2019s fun. But going back in processor generations just doesn\u2019t feel great. I want something more efficient, not less! And then I discovered the Pixelbook. Other than the big huge large bezels around the screen, I liked everything about it. Thin aluminum design, a 3:2 HiDPI screen, rubber palm rests (why isn\u2019t every laptop ever doing that?!), the \u201cconvertibleness\u201d (flip the screen around to turn it into.. something rather big for a tablet, but it is useful actually), a Wacom touchscreen that supports a pen, mostly reasonable hardware (Intel Wi-Fi), and that famous coreboot support (Chromebooks\u2019 stock firmware is coreboot + depthcharge). So here it is, my new laptop, a Google Pixelbook. Conclusion Pixelbook, FreeBSD, coreboot, EDK2 good. Seriously, I have no big words to say, other than just recommending this laptop to FOSS enthusiasts :) Porting NetBSD to the AMD x86-64: a case study in OS portability (https://www.usenix.org/legacy/publications/library/proceedings/bsdcon02/full_papers/linden/linden_html/index.html) Abstract NetBSD is known as a very portable operating system, currently running on 44 different architectures (12 different types of CPU). This paper takes a look at what has been done to make it portable, and how this has decreased the amount of effort needed to port NetBSD to a new architecture. The new AMD x86-64 architecture, of which the specifications were published at the end of 2000, with hardware to follow in 2002, is used as an example. Portability Supporting multiple platforms was a primary goal of the NetBSD project from the start. As NetBSD was ported to more and more platforms, the NetBSD kernel code was adapted to become more portable along the way. General Generally, code is shared between ports as much as possible. In NetBSD, it should always be considered if the code can be assumed to be useful on other architectures, present or future. If so, it is machine-independent and put it in an appropriate place in the source tree. When writing code that is intended to be machine-independent, and it contains conditional preprocessor statements depending on the architecture, then the code is likely wrong, or an extra abstraction layer is needed to get rid of these statements. Types Assumptions about the size of any type are not made. Assumptions made about type sizes on 32-bit platforms were a large problem when 64-bit platforms came around. Most of the problems of this kind had to be dealt with when NetBSD was ported to the DEC Alpha in 1994. A variation on this problem had to be dealt with with the UltraSPARC (sparc64) port in 1998, which is 64-bit, but big endian (vs. the little-endianness of the Alpha). When interacting with datastructures of a fixed size, such as on-disk metadata for filesystems, or datastructures directly interpreted by device hardware, explicitly sized types are used, such as uint32t, int8t, etc. Conclusions and future work The port of NetBSD to AMD's x86-64 architecture was done in six weeks, which confirms NetBSD's reputation as being a very portable operating system. One week was spent setting up the cross-toolchain and reading the x86-64 specifications, three weeks were spent writing the kernel code, one week was spent writing the userspace code, and one week testing and debugging it all. No problems were observed in any of the machine-independent parts of the kernel during test runs; all (simulated) device drivers, file systems, etc, worked without modification. News Roundup ZFS performance really does degrade as you approach quota limits (https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSFullQuotaPerformanceIssue) Every so often (currently monthly), there is an "OpenZFS leadership meeting". What this really means is 'lead developers from the various ZFS implementations get together to talk about things'. Announcements and meeting notes from these meetings get sent out to various mailing lists, including the ZFS on Linux ones. In the September meeting notes, I read a very interesting (to me) agenda item: Relax quota semantics for improved performance (Allan Jude) Problem: As you approach quotas, ZFS performance degrades. Proposal: Can we have a property like quota-policy=strict or loose, where we can optionally allow ZFS to run over the quota as long as performance is not decreased. This is very interesting to me because of two reasons. First, in the past we have definitely seen significant problems on our OmniOS machines, both when an entire pool hits a quota limit and when a single filesystem hits a refquota limit. It's nice to know that this wasn't just our imagination and that there is a real issue here. Even better, it might someday be improved (and perhaps in a way that we can use at least some of the time). Second, any number of people here run very close to and sometimes at the quota limits of both filesystems and pools, fundamentally because people aren't willing to buy more space. We have in the past assumed that this was relatively harmless and would only make people run out of space. If this is a known issue that causes serious performance degradation, well, I don't know if there's anything we can do, but at least we're going to have to think about it and maybe push harder at people. The first step will have to be learning the details of what's going on at the ZFS level to cause the slowdown. (It's apparently similar to what happens when the pool is almost full, but I don't know the specifics of that either.) With that said, we don't seem to have seen clear adverse effects on our Linux fileservers, and they've definitely run into quota limits (repeatedly). One possible reason for this is that having lots of RAM and SSDs makes the effects mostly go away. Another possible reason is that we haven't been looking closely enough to see that we're experiencing global slowdowns that correlate to filesystems hitting quota limits. We've had issues before with somewhat subtle slowdowns that we didn't understand (cf), so I can't discount that we're having it happen again. Fixing up KA9Q-unix, or "neck deep in 30 year old codebases.." (http://adrianchadd.blogspot.com/2019/09/fixing-up-ka9q-unix-or-neck-deep-in-30.html) I'll preface this by saying - yes, I'm still neck deep in FreeBSD's wifi stack and 802.11ac support, but it turns out it's slow work to fix 15 year old locking related issues that worked fine on 11abg cards, kinda worked ok on 11n cards, and are terrible for these 11ac cards. I'll .. get there. Anyhoo, I've finally been mucking around with AX.25 packet radio. I've been wanting to do this since I was a teenager and found out about its existence, but back in high school and .. well, until a few years ago really .. I didn't have my amateur radio licence. But, now I do, and I've done a bunch of other stuff with a bunch of other radios. The main stumbling block? All my devices are either Apple products or run FreeBSD - and none of them have useful AX.25 stacks. The main stacks of choice these days run on Linux, Windows or are a full hardware TNC. So yes, I was avoiding hacking on AX.25 stuff because there wasn't a BSD compatible AX.25 stack. I'm 40 now, leave me be. But! A few weeks ago I found that someone was still running a packet BBS out of San Francisco. And amazingly, his local node ran on FreeBSD! It turns out Jeremy (KK6JJJ) ported both an old copy of KA9Q and N0ARY-BBS to run on FreeBSD! Cool! I grabbed my 2m radio (which is already cabled up for digital modes), compiled up his KA9Q port, figured out how to get it to speak to Direwolf, and .. ok. Well, it worked. Kinda. HAMMER2 and fsck for review (https://www.dragonflydigest.com/2019/09/24/23540.html) HAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there\u2019s now a fsck command, useful if you want a report of data validity rather than any manual repair process. [The return of startx(1) for non-root users with some caveats (https://undeadly.org/cgi?action=article;sid=20190917091236) Mark Kettenis (kettenis@) has recently committed changes which restore a certain amount of startx(1)/xinit(1) functionality for non-root users. The commit messages explain the situation: ``` CVSROOT: /cvs Module name: src Changes by: kettenis@cvs.openbsd.org 2019/09/15 06:25:41 Modified files: etc/etc.amd64 : fbtab etc/etc.arm64 : fbtab etc/etc.hppa : fbtab etc/etc.i386 : fbtab etc/etc.loongson: fbtab etc/etc.luna88k: fbtab etc/etc.macppc : fbtab etc/etc.octeon : fbtab etc/etc.sgi : fbtab etc/etc.sparc64: fbtab Log message: Add ttyC4 to lost of devices to change when logging in on ttyC0 (and in some cases also the serial console) such that X can use it as its VT when running without root privileges. ok jsg@, matthieu@ CVSROOT: /cvs Module name: xenocara Changes by: kettenis@cvs.openbsd.org 2019/09/15 06:31:08 Modified files: xserver/hw/xfree86/common: xf86AutoConfig.c Log message: Add modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus. This makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4). In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)). ok jsg@, matthieu@ ``` Beastie Bits ASCII table and history. Or, why does Ctrl+i insert a Tab in my terminal? (https://bestasciitable.com/) Sourcehut makes BSD software better (https://sourcehut.org/blog/2019-09-12-sourcehut-makes-bsd-software-better/) Chaosnet for Unx (https://github.com/LM-3/chaos) The Vim-Inspired Editor with a Linguistic Twist (https://cosine.blue/2019-09-06-kakoune.html) bhyvearm64: CPU and Memory Virtualization on Armv8.0-A (https://papers.freebsd.org/2019/bsdcan/elisei-bhyvearm64_cpu_and_memory_virtualization_on_armv8.0_a/) DefCon25 - Are all BSD created Equally - A Survey of BSD Kernel vulnerabilities (https://www.youtube.com/watch?v=a2m56Yq-EIs) Feedback/Questions Tim - GSoC project ideas for pf rule syntax translation (http://dpaste.com/1RCSFK7#wrap) Brad - Steam on FreeBSD (http://dpaste.com/2SKA9YB#wrap) Ruslan - FreeBSD Quarterly Status Report - Q2 2019 (http://dpaste.com/0DQM3Q1) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 16 Oct 2019 20:00:00 -0700
319: Lack Rack, Jack
Causing ZFS corruption for fun, NetBSD Assembly Programming Tutorial, The IKEA Lack Rack for Servers, a new OmniOS Community Edition LTS has been published, List Block Devices on FreeBSD lsblk(8) Style, Project Trident 19.10 available, and more. Headlines Causing ZFS corruption for fun and profit (https://datto.engineering/post/causing-zfs-corruption) Datto backs up data, a lot of it. At the time of writing Datto has over 500 PB of data stored on ZFS. This count includes both backup appliances that are sent to customer sites, as well as cloud storage servers that are used for secondary and tertiary backup of those appliances. At this scale drive swaps are a daily occurrence, and data corruption is inevitable. How we handle this corruption when it happens determines whether we truly lose data, or successfully restore from secondary backup. In this post we'll be showing you how at Datto we intentionally cause corruption in our testing environments, to ensure we're building software that can properly handle these scenarios. Causing Corruption Since this is a mirror setup, a naive solution to cause corruption would be to randomly dd the same sectors of both /dev/sdb and /dev/sdc. This works, but is equally likely to just overwrite random unused space, or take down the zpool entirely. What we really want is to corrupt a specific snapshot, or even a specific file in that snapshot, to simulate a more realistic minor corruption event. Luckily we have a tool called zdb that lets us view some low level information about datasets. Conclusion At the 500 PB scale, it's not a matter of if data corruption will happen but when. Intentionally causing corruption is one of the strategies we use to ensure we're building software that can handle these rare (but inevitable) events. To others out there using ZFS: I'm curious to hear how you've solved this problem. We did quite a bit of experimentation with zinject before going with this more brute force method. So I'd be especially interested if you've had luck simply simulating corruption with zinject. NetBSD Assembly Programming Tutorial (https://polprog.net/blog/netbsdasmprog/) A sparc64 version is also being prepared and will be added when done This post describes how to write a simple hello world program in pure assembly on NetBSD/amd64. We will not use (nor link against) libc, nor use gcc to compile it. I will be using GNU as (gas), and therefore the AT&T syntax instead of Intel. Why assembly? Why not? Because it's fun to program in assembly directly. Contrary to a popular belief assembly programs aren't always faster than what optimizing compilers produce. Nevertheless it's good to be able to read assembly, especially when debugging C programs Due to the nature of the guide, visit the site for the complete breakdown News Roundup The IKEA Lack Rack for Servers (https://wiki.eth0.nl/index.php/LackRack) The LackRack First occurrence on eth0:2010 Winterlan, the LackRack is the ultimate, low-cost, high shininess solution for your modular datacenter-in-the-living-room. Featuring the LACK (side table) from Ikea, the LackRack is an easy-to-implement, exact-fit datacenter building block. It's a little known fact that we have seen Google engineers tinker with Lack tables since way back in 2009. The LackRack will certainly make its appearance again this summer at eth0:2010 Summer. Summary When temporarily not in use, multiple LackRacks can be stacked in a space-efficient way without disassembly, unlike competing 19" server racks. The LackRack was first seen on eth0:2010 Winterlan in the no-shoe Lounge area. Its low-cost and perfect fit are great for mounting up to 8 U of 19" hardware, such as switches (see below), or perhaps other 19" gear. It's very easy to assemble, and thanks to the design, they are stable enough to hold (for example) 19" switches and you can put your bottle of Club-Mate on top! Multi-shiny LackRack can also be painted to your specific preferences and the airflow is unprecedented! Howto You can find a howto on buying a LackRack on this page. This includes the proof that a 19" switch can indeed be placed in the LackRack in its natural habitat! OmniOS Community Edition r151030 LTS - Published at May 6, 2019 (https://omniosce.org/article/release-030) The OmniOS Community Edition Association is proud to announce the general availability of OmniOS - r151030. OmniOS is published according to a 6-month release cycle, r151030 LTS takes over from r151028, published in November 2018; and since it is a LTS release it also takes over from r151022. The r151030 LTS release will be supported for 3 Years. It is the first LTS release published by the OmniOS CE Association since taking over the reins from OmniTI in 2017. The next LTS release is scheduled for May 2021. The old stable r151026 release is now end-of-life. See the release schedule for further details. This is only a small selection of the new features, and bug fixes in the new release; review the release notes for full details. If you upgrade from r22 and want to see all new features added since then, make sure to also read the release notes for r24, r26 and r28. For full relase notes including upgrade instructions; release notes (https://omniosce.org/releasenotes.html) upgrade instructions (https://omniosce.org/upgrade.html) List Block Devices on FreeBSD lsblk(8) Style (https://vermaden.wordpress.com/2019/09/27/list-block-devices-on-freebsd-lsblk8-style/) When I have to work on Linux systems I usually miss many nice FreeBSD tools such as these for example to name the few: sockstat, gstat, top -b -o res, top -m io -o total, usbconfig, rcorder, beadm/bectl, idprio/rtprio,… but sometimes – which rarely happens – Linux has some very useful tool that is not available on FreeBSD. An example of such tool is lsblk(8) that does one thing and does it quite well – lists block devices and their contents. It has some problems like listing a disk that is entirely used under ZFS pool on which lsblk(8) displays two partitions instead of information about ZFS just being there – but we all know how much in some circles the CDDL licensed ZFS is unloved in that GPL world. Example lsblk(8) output from Linux system: $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 1024M 0 rom sda 8:0 0 931.5G 0 disk |-sda1 8:1 0 500M 0 part /boot `-sda2 8:2 0 931G 0 part |-vg_local-lv_root (dm-0) 253:0 0 50G 0 lvm / |-vg_local-lv_swap (dm-1) 253:1 0 17.7G 0 lvm [SWAP] `-vg_local-lv_home (dm-2) 253:2 0 1.8T 0 lvm /home sdc 8:32 0 232.9G 0 disk `-sdc1 8:33 0 232.9G 0 part `-md1 9:1 0 232.9G 0 raid10 /data sdd 8:48 0 232.9G 0 disk `-sdd1 8:49 0 232.9G 0 part `-md1 9:1 0 232.9G 0 raid10 /data What FreeBSD offers in this department? The camcontrol(8) and geom(8) commands are available. You can also use gpart(8) command to list partitions. Below you will find output of these commands from my single disk laptop. Please note that because of WordPress limitations I need to change all > < characters to ] [ ones in the commands outputs. See the article for the rest of the guide Project Trident 19.10 Now Available (https://project-trident.org/post/2019-10-05_19.10_available/) This is a general package update to the CURRENT release repository based upon TrueOS 19.10 PACKAGE CHANGES FROM 19.08 New Packages: 601 Deleted Packages: 165 Updated Packages: 3341 Beastie Bits NetBSD building tools (https://imgur.com/gallery/0sG4b1K) Sponsorships open for SNMP Mastery (https://mwl.io/archives/4569) pkgsrc-2019Q3 release announcement (2019-10-03) (http://mail-index.netbsd.org/pkgsrc-users/2019/10/03/msg029485.html) pfetch - A simple system information tool written in POSIX sh (https://github.com/dylanaraps/pfetch) Taking NetBSD kernel bug roast to the next level: Kernel Fuzzers (quick A.D. 2019 overview) (https://netbsd.org/~kamil/eurobsdcon2019_fuzzing/presentation.html#slide1) Cracking Ken Thomson’s password (https://leahneukirchen.org/blog/archive/2019/10/ken-thompson-s-unix-password.html) Feedback/Questions Evilham - Couple Questions (http://dpaste.com/2JC85WV) Rob - APU2 alternatives and GPT partition types (http://dpaste.com/0SDX9ZX) Tom - FreeBSD journal article by A. Fengler (http://dpaste.com/2B43MY1#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 09 Oct 2019 20:00:00 -0700
318: The TrueNAS Library
DragonFlyBSD vs. FreeBSD vs. Linux benchmark on Ryzen 7, JFK Presidential Library chooses TrueNAS for digital archives, FreeBSD 12.1-beta is available, cool but obscure X11 tools, vBSDcon trip report, Project Trident 12-U7 is available, a couple new Unix artifacts, and more. Headlines DragonFlyBSD 5.6 vs. FreeBSD 12 vs. Linux - Ryzen 7 3700X (https://www.phoronix.com/scan.php?page=article&item=bsd-linux-3700x) For those wondering how well FreeBSD and DragonFlyBSD are handling AMD's new Ryzen 3000 series desktop processors, here are some benchmarks on a Ryzen 7 3700X with MSI MEG X570 GODLIKE where both of these popular BSD operating systems were working out-of-the-box. For some fun mid-week benchmarking, here are those results of FreeBSD 12.0 and DragonFlyBSD 5.6.2 up against openSUSE Tumbleweed and Ubuntu 19.04. Back in July I looked at FreeBSD 12 on the Ryzen 9 3900X but at that time at least DragonFlyBSD had troubles booting on that system. When trying out the Ryzen 7 3700X + MSI GODLIKE X570 motherboard on the latest BIOS, everything "just worked" without any compatibility issues for either of these BSDs. We've been eager to see how well DragonFlyBSD is performing on these new AMD Zen 2 CPUs with DragonFlyBSD lead developer Matthew Dillon having publicly expressed being impressed by the new AMD Ryzen 3000 series CPUs. For comparison to those BSDs, Ubuntu 19.04 and openSUSE Tumbleweed were tested on the same hardware in their out-of-the-box configurations. While Clear Linux is normally the fastest, on this system Clear's power management defaults had caused issues in being unable to detect the Samsung 970 EVO Plus NVMe SSD used for testing and so we left it out this round. All of the hardware was the same throughout testing as were the BIOS settings and running the Ryzen 7 3700X at stock speeds. (Any differences in the reported hardware for the system table just come down to differences in what is exposed by each OS for reporting.) All of the BSD/Linux benchmarks on this eight core / sixteen thread processor were run via the Phoronix Test Suite. In the case of FreeBSD 12.0, we benchmarked both with its default LLVM Clang 6.0 compiler as well as with GCC 9.1 so that it would match the GCC compiler being the default on the other operating systems under test. JFK Presidential Library Chooses iXsystems TrueNAS to Preserve Precious Digital Archives (https://www.ixsystems.com/blog/jfk-presidential-library-pr/) iXsystems is honored to have the TrueNAS® M-Series unified storage selected to store, serve, and protect the entire digital archive for the John F. Kennedy Library Foundation. This is in support of the collection at the John F. Kennedy Presidential Library and Museum (JFK Library). Over the next several years, the Foundation hopes to grow the digital collection from hundreds of terabytes today to cover much more of the Archives at the Kennedy Library. Overall there is a total of 25 million documents, audio recordings, photos, and videos once the project is complete. Having first deployed the TrueNAS M50-HA earlier in 2019, the JFK Library has now completed the migration of its existing digital collection and is now in the process of digitizing much of the rest of its vast collection. Not only is the catalog of material vast, it is also diverse, with files being copied to the storage system from a variety of sources in numerous file types. To achieve this ambitious goal, the library required a high-end NAS system capable of sharing with a variety of systems throughout the digitization process. The digital archive will be served from the TrueNAS M50 and made available to both in-person and online visitors. With precious material and information comes robust demands. The highly-available TrueNAS M-Series has multiple layers of protection to help keep data safe, including data scrubs, checksums, unlimited snapshots, replication, and more. TrueNAS is also inherently scalable with data shares only limited by the number of drives connected to the pool. Perfect for archival storage, the deployed TrueNAS M50 will grow with the library’s content, easily expanding its storage capacity over time as needed. Supporting a variety of protocols, multi-petabyte scalability in a single share, and anytime, uninterrupted capacity expansion, the TrueNAS M-Series ticked all the right boxes. Youtube Video (https://www.youtube.com/watch?v=8rFjH5-0Fiw) News Roundup FreeBSD 12.1-beta available (https://www.phoronix.com/scan.php?page=news_item&px=FreeBSD-12.1-Beta-Released) FreeBSD 12.0 is already approaching one year old while FreeBSD 12.1 is now on the way as the next installment with various bug/security fixes and other alterations to this BSD operating system. FreeBSD 12.1 has many security/bug fixes throughout, no longer enables "-Werror" by default as a compiler flag (Update: This change is just for the GCC 4.2 compiler), has imported BearSSL into the FreeBSD base system as a lightweight TLS/SSL implementation, bzip2recover has been added, and a variety of mostly lower-level changes. More details can be found via the in-progress release notes. For those with time to test this weekend, FreeBSD 12.1 Beta 1 is available for all prominent architectures. The FreeBSD release team is planning for at least another beta or two and around three release candidates. If all goes well, FreeBSD 12.1 will be out in early November. Announcement Link (https://lists.freebsd.org/pipermail/freebsd-stable/2019-September/091533.html) Cool, but obscure X11 tools. More suggestions in the source link (https://cyber.dabamos.de/unix/x11/) ASClock Free42 FSV2 GLXGears GMixer GVIM Micropolis Sunclock Ted TiEmu X026 X48 XAbacus XAntfarm XArchiver XASCII XBiff XBill XBoard XCalc XCalendar XCHM XChomp XClipboard XClock XClock/Cat Clock XColorSel XConsole XDiary XEarth XEdit Xev XEyes XFontSel XGalaga XInvaders 3D XKill XLennart XLoad XLock XLogo XMahjongg XMan XMessage XmGrace XMixer XmMix XMore XMosaic XMOTD XMountains XNeko XOdometer XOSView Xplore XPostIt XRoach XScreenSaver XSnow XSpread XTerm XTide Xv Xvkbd XWPE XZoom vBSDCon 2019 trip report from iXSystems (https://www.ixsystems.com/blog/vbsdcon-2019/) The fourth biennial vBSDCon was held in Reston, VA on September 5th through 7th and attracted attendees and presenters from not only the Washington, DC area, but also Canada, Germany, Kenya, and beyond. While MeetBSD caters to Silicon Valley BSD enthusiasts on even years, vBSDcon caters to East Coast and DC area enthusiasts on odd years. Verisign was again the key sponsor of vBSDcon 2019 but this year made a conscious effort to entrust the organization of the event to a team of community members led by Dan Langille, who you probably know as the lead BSDCan organizer. The result of this shift was a low key but professional event that fostered great conversation and brainstorming at every turn. Project Trident 12-U7 now available (https://project-trident.org/post/2019-09-21_stable12-u7_available/) Package Summary New Packages: 130 Deleted Packages: 72 Updated Packages: 865 Stable ISO - https://pkg.project-trident.org/iso/stable/Trident-x64-TOS-12-U7-20190920.iso A Couple new Unix Artifacts (https://minnie.tuhs.org//pipermail/tuhs/2019-September/018685.html) I fear we're drifting a bit here and the S/N ratio is dropping a bit w.r.t the actual history of Unix. Please no more on the relative merits of version control systems or alternative text processing systems. So I'll try to distract you by saying this. I'm sitting on two artifacts that have recently been given to me: by two large organisations of great significance to Unix history who want me to keep "mum" about them as they are going to make announcements about them soon* and I am going slowly crazy as I wait for them to be offically released. Now you have a new topic to talk about :-) Cheers, Warren * for some definition of "soon" Beastie Bits NetBSD machines at Open Source Conference 2019 Hiroshima (https://mail-index.netbsd.org/netbsd-advocacy/2019/09/16/msg000813.html) Hyperbola a GNU/Linux OS is using OpenBSD's Xenocara (https://www.hyperbola.info/news/end-of-xorg-support/) Talos is looking for a FreeBSD Engineer (https://www.talosintelligence.com/careers/freebsd_engineer) GitHub - dylanaraps/pure-sh-bible: A collection of pure POSIX sh alternatives to external processes. (https://github.com/dylanaraps/pure-sh-bible) dsynth: you’re building it (https://www.dragonflydigest.com/2019/09/23/23523.html) Percy Ludgate, the missing link between Babbage’s machine and everything else (http://lists.sigcis.org/pipermail/members-sigcis.org/2019-September/001606.html) Feedback/Questions Bruce - Down the expect rabbithole (http://dpaste.com/147HGP3#wrap) Bruce - Expect (update) (http://dpaste.com/37MNVSW#wrap) David - Netgraph answer (http://dpaste.com/2SE1YSE) Mason - Beeps? (http://dpaste.com/00KKXJM) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 02 Oct 2019 20:00:00 -0700
317: Bots Building Jails
Setting up buildbot in FreeBSD jails, Set up a mail server with OpenSMTPD, Dovecot and Rspamd, OpenBSD amateur packet radio with HamBSD, DragonFlyBSD's HAMMER2 gets fsck, return of startx for users. Headlines EuroBSDcon 2019 Recap (https://2019.eurobsdcon.org/) We’re back from EuroBSDcon in Lillehammer, Norway. It was a great conference with 212 people attending. 2 days of tutorials (https://2019.eurobsdcon.org/tutorial-speakers/), parallel to the FreeBSD Devsummit (https://wiki.freebsd.org/DevSummit/201909), followed by two days of talks (https://2019.eurobsdcon.org/program/). Some speakers uploaded their slides to papers.freebsd.org (https://papers.freebsd.org/2019/eurobsdcon/) already with more to come. The social event was also interesting. We visited an open air museum with building preserved from different time periods. In the older section they had a collection of farm buildings, a church originally built in the 1200s and relocated to the museum, and a school house. In the more modern area, they had houses from 1915, and each decade from 1930 to 1990, plus a “house of the future” as imagined in 2001. Many had open doors to allow you to tour the inside, and some were even “inhabited”. The latter fact gave a much more interactive experience and we could learn additional things about the history of that particular house. The town at the end included a general store, a post office, and more. Then, we all had a nice dinner together in the museum’s restaurant. The opening keynote by Patricia Aas was very good. Her talk on embedded ethics, from her perspective as someone trying to defend the sanctity of Norwegian elections, and a former developer for the Opera web browser, provided a great deal of insight into the issues. Her points about how the tech community has unleashed a very complex digital work upon people with barely any technical literacy were well taken. Her stories of trying to explain the problems with involving computers in the election process to journalists and politicians struck a chord with many of us, who have had to deal with legislation written by those who do not truly understand the issues with technology. Setting up buildbot in FreeBSD jails (https://andidog.de/blog/2018-04-22-buildbot-setup-freebsd-jails) In this article, I would like to present a tutorial to set up buildbot, a continuous integration (CI) software (like Jenkins, drone, etc.), making use of FreeBSD’s containerization mechanism "jails". We will cover terminology, rationale for using both buildbot and jails together, and installation steps. At the end, you will have a working buildbot instance using its sample build configuration, ready to play around with your own CI plans (or even CD, it’s very flexible!). Some hints for production-grade installations are given, but the tutorial steps are meant for a test environment (namely a virtual machine). Buildbot’s configuration and detailed concepts are not in scope here. Setting up a mail server with OpenSMTPD, Dovecot and Rspamd (https://poolp.org/posts/2019-09-14/setting-up-a-mail-server-with-opensmtpd-dovecot-and-rspamd/) Self-hosting and encouraging smaller providers is for the greater good First of all, I was not clear enough about the political consequences of centralizing mail services at Big Mailer Corps. It doesn’t make sense for Random Joe, sharing kitten pictures with his family and friends, to build a personal mail infrastructure when multiple Big Mailer Corps offer “for free” an amazing quality of service. They provide him with an e-mail address that is immediately available and which will generally work reliably. It really doesn’t make sense for Random Joe not to go there, and particularly if even techies go there without hesitation, proving it is a sound choice. There is nothing wrong with Random Joes using a service that works. What is terribly wrong though is the centralization of a communication protocol in the hands of a few commercial companies, EVERY SINGLE ONE OF THEM coming from the same country (currently led by a lunatic who abuses power and probably suffers from NPD), EVERY SINGLE ONE OF THEM having been in the news and/or in a court for random/assorted “unpleasant” behaviors (privacy abuses, eavesdropping, monopoly abuse, sexual or professional harassment, you just name it…), and EVERY SINGLE ONE OF THEM growing user bases that far exceeds the total population of multiple countries combined. News Roundup The HamBSD project aims to bring amateur packet radio to OpenBSD (https://hambsd.org/) The HamBSD project aims to bring amateur packet radio to OpenBSD, including support for TCP/IP over AX.25 and APRS tracking/digipeating in the base system. HamBSD will not provide a full AX.25 stack but instead only implement support for UI frames. There will be a focus on simplicity, security and readable code. The amateur radio community needs a reliable platform for packet radio for use in both leisure and emergency scenarios. It should be expected that the system is stable and resilient (but as yet it is neither). DragonFlyBSD's HAMMER2 Gets Basic FSCK Support (https://www.dragonflydigest.com/2019/09/24/23540.html) HAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there’s now a fsck command, useful if you want a report of data validity rather than any manual repair process. commit (https://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/5554cc8b81fbfcfd347f50be3f3b1b9a54b871b) Add initial fsck support for HAMMER2, although CoW fs doesn't require fsck as a concept. Currently no repairing (no write), just verifying. Keep this as a separate command for now. https://i.redd.it/vkdss0mtdpo31.jpg The return of startx for users (http://undeadly.org/cgi?action=article;sid=20190917091236) Add modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus. This makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4). In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)). Beastie Bits Ori Bernstein will be giving the October talk at NYCBUG (http://lists.nycbug.org:8080/pipermail/talk/2019-September/018046.html) BSD Pizza Night: 2019/09/26, 7–9PM, Portland, Oregon, USA (http://calagator.org/events/1250476200) Nick Wolff : Home Lab Show & Tell (http://knoxbug.org/2019-09-30) Installing the Lumina Desktop in DragonflyBSD (https://www.youtube.com/watch?v=eWkCjj4_xsk) dhcpcd 8.0.6 added (https://www.dragonflydigest.com/2019/09/20/23519.html) Feedback/Questions Bruce - FOSDEM videos (http://dpaste.com/15ABRRB#wrap) Lars - Super Cluster of BSD on Rock64Pr (http://dpaste.com/1X9FEJJ) Madhukar - Question (http://dpaste.com/0TWF1NB#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 25 Sep 2019 23:00:00 -0700
316: git commit FreeBSD
NetBSD LLVM sanitizers and GDB regression test suite, Ada—The Language of Cost Savings, Homura - a Windows Games Launcher for FreeBSD, FreeBSD core team appoints a WG to explore transition to Git, OpenBSD 6.6 Beta tagged, Project Trident 12-U5 update now available, and more. Headlines LLVM santizers and GDB regression test suite. (http://blog.netbsd.org/tnf/entry/llvm_santizers_and_gdb_regression) As NetBSD-9 is branched, I have been asked to finish the LLVM sanitizer integration. This work is now accomplished and with MKLLVM=yes build option (by default off), the distribution will be populated with LLVM files for ASan, TSan, MSan, UBSan, libFuzzer, SafeStack and XRay. I have also transplanted basesystem GDB patched to my GDB repository and managed to run the GDB regression test-suite. NetBSD distribution changes I have enhanced and imported my local MKSANITIZER code that makes whole distribution sanitization possible. Few real bugs were fixed and a number of patches were newly written to reflect the current NetBSD sources state. I have also merged another chunk of the fruits of the GSoC-2018 project with fuzzing the userland (by plusun@). The following changes were committed to the sources: ab7de18d0283 Cherry-pick upstream compiler-rt patches for LLVM sanitizers 966c62a34e30 Add LLVM sanitizers in the MKLLVM=yes build 8367b667adb9 telnetd: Stop defining the same variables concurrently in bss and data fe72740f64bf fsck: Stop defining the same variable concurrently in bss and data 40e89e890d66 Fix build of tubsan/tubsanxx under MKSANITIZER b71326fd7b67 Avoid symbol clashes in tests/usr.bin/id under MKSANITIZER c581f2e39fa5 Avoid symbol clashes in fs/nfs/nfsservice under MKSANITIZER 030a4686a3c6 Avoid symbol clashes in bin/df under MKSANITIZER fd9679f6e8b1 Avoid symbol clashes in usr.sbin/ypserv/ypserv under MKSANITIZER 5df2d7939ce3 Stop defining _rpcsvcdirty in bss and data 5fafbe8b8f64 Add missing extern declaration of ibmachemips in installboot d134584be69a Add SANITIZERRENAMECLASSES in bsd.prog.mk 2d00d9b08eae Adapt tests/kernel/tsubrprf for MKSANITIZER ce54363fe452 Ship with sanitizer/lsan_interface.h for GCC 7 7bd5ee95e9a0 Ship with sanitizer/lsan_interface.h for LLVM 7 d8671fba7a78 Set NODEBUG for LLVM sanitizers 242cd44890a2 Add PAXCTL_FLAG rules for MKSANITIZER 5e80ab99d9ce Avoid symbol clashes in test/rump/modautoload/t_modautoload with sanitizers e7ce7ecd9c2a sysctl: Add indirection of symbols to remove clash with sanitizers 231aea846aba traceroute: Add indirection of symbol to remove clash with sanitizers 8d85053f487c sockstat: Add indirection of symbols to remove clash with sanitizers 81b333ab151a netstat: Add indirection of symbols to remove clash with sanitizers a472baefefe8 Correct the memset(3)'s third argument in i386 biosdisk.c 7e4e92115bc3 Add ATF c and c++ tests for TSan, MSan, libFuzzer 921ddc9bc97c Set NOSANITIZER in i386 ramdisk image 64361771c78d Enhance MKSANITIZER support 3b5608f80a2b Define targetnotsupported_body() in TSan, MSan and libFuzzer tests c27f4619d513 Avoids signedness bit shift in dbgetvalue() 680c5b3cc24f Fix LLVM sanitizer build by GCC (HAVE_LLVM=no) 4ecfbbba2f2a Rework the LLVM compiler_rt build rules 748813da5547 Correct the build rules of LLVM sanitizers 20e223156dee Enhance the support of LLVM sanitizers 0bb38eb2f20d Register syms.extra in LLVM sanitizer .syms files Almost all of the mentioned commits were backported to NetBSD-9 and will land 9.0. Homura - a Windows Games Launcher for FreeBSD (https://github.com/Alexander88207/Homura) Inspired by lutris (a Linux gaming platform), we would like to provide a game launcher to play windows games on FreeBSD. Makes it easier to run games on FreeBSD, by providing the tweaks and dependencies for you Dependencies curl bash p7zip zenity webfonts alsa-utils (Optional) winetricks vulkan-tools mesa-demos i386-wine-devel on amd64 or wine-devel on i386 News Roundup Ada—The Language of Cost Savings? (https://www.electronicdesign.com/embedded-revolution/ada-language-cost-savings) Many myths surround the Ada programming language, but it continues to be used and evolve at the same time. And while the increased adoption of Ada and SPARK, its provable subset, is slow, it’s noticeable. Ada already addresses more of the features found in found in heavily used embedded languages like C+ and C#. It also tackles problems addressed by upcoming languages like Rust. Chris concludes, “Development technologies have a profound impact on one of the largest and most variable costs associated with embedded-system engineering—labor. At a time when on-time system deployment can not only impact customer satisfaction, but access to services revenue streams, engineering team efficiency is at a premium. Our research showed that programming language choices can have significant influence in this area, leading to shorter projects, better schedules and, ultimately, lower development costs. While a variety of factors can influence and dictate language choice, our research showed that Ada’s evolution has made it an increasingly compelling option for engineering organizations, providing both technically and financially sound solution.” In general, Ada already makes embedded “programming in the large” much easier by handling issues that aren’t even addressed in other languages. Though these features are often provided by third-party software, it results in inconsistent practices among developers. Ada also supports the gamut of embedded platforms from systems like Arm’s Cortex-M through supercomputers. Learning Ada isn’t as hard as one might think and the benefits can be significant. FreeBSD core team appoints a WG to explore transitioning from Subversion to Git. (https://www.freebsd.org/news/status/report-2019-04-2019-06.html#FreeBSD-Core-Team) The FreeBSD Core Team is the governing body of FreeBSD. Core approved source commit bits for Doug Moore (dougm), Chuck Silvers (chs), Brandon Bergren (bdragon), and a vendor commit bit for Scott Phillips (scottph). The annual developer survey closed on 2019-04-02. Of the 397 developers, 243 took the survey with an average completion time of 12 minutes. The public survey closed on 2019-05-13. It was taken by 3637 users and had a 79% completion rate. A presentation of the survey results took place at BSDCan 2019. The core team voted to appoint a working group to explore transitioning our source code 'source of truth' from Subversion to Git. Core asked Ed Maste to chair the group as Ed has been researching this topic for some time. For example, Ed gave a MeetBSD 2018 talk on the topic. There is a variety of viewpoints within core regarding where and how to host a Git repository, however core feels that Git is the prudent path forward. OpenBSD 6.6 Beta tagged (https://undeadly.org/cgi?action=article;sid=20190810123243) ``` CVSROOT: /cvs Module name: src Changes by: deraadt@cvs.openbsd.org 2019/08/09 21:56:02 Modified files: etc/root : root.mail share/mk : sys.mk sys/arch/macppc/stand/tbxidata: bsd.tbxi sys/conf : newvers.sh sys/sys : param.h usr.bin/signify: signify.1 Log message: move to 6.6-beta ``` Preliminary release notes (https://www.openbsd.org/66.html) Improved hardware support, including: clang(1) is now provided on powerpc. IEEE 802.11 wireless stack improvements: Generic network stack improvements: Installer improvements: Security improvements: + Routing daemons and other userland network improvements + The ntpd(8) daemon now gets and sets the clock in a secure way when booting even when a battery-backed clock is absent. + bgdp(8) improvements + Assorted improvements: + The filesystem buffer cache now more aggressively uses memory outside the DMA region, to improve cache performance on amd64 machines. The BER API previously internal to ldap(1), ldapd(8), ypldap(8), and snmpd(8) has been moved into libutil. See berreadelements(3). Support for specifying boot device in vm.conf(5). OpenSMTPD 6.6.0 LibreSSL 3.0.X API and Documentation Enhancements Completed the port of RSA_METHOD accessors from the OpenSSL 1.1 API. Documented undescribed options and removed unfunctional options description in openssl(1) manual. OpenSSH 8.0 Project Trident 12-U5 update now available (https://project-trident.org/post/2019-09-04_stable12-u5_available/) This is the fifth general package update to the STABLE release repository based upon TrueOS 12-Stable. Package changes from Stable 12-U4 Package Summary New Packages: 20 Deleted Packages: 24 Updated Packages: 279 New Packages (20) artemis (biology/artemis) : 17.0.1.11 catesc (games/catesc) : 0.6 dmlc-core (devel/dmlc-core) : 0.3.105 go-wtf (sysutils/go-wtf) : 0.20.0_1 instead (games/instead) : 3.3.0_1 lidarr (net-p2p/lidarr) : 0.6.2.883 minerbold (games/minerbold) : 1.4 onnx (math/onnx) : 1.5.0 openzwave-devel (comms/openzwave-devel) : 1.6.897 polkit-qt-1 (sysutils/polkit-qt) : 0.113.0_8 py36-traitsui (graphics/py-traitsui) : 6.1.2 rubygem-aws-sigv2 (devel/rubygem-aws-sigv2) : 1.0.1 rubygem-defaultvaluefor32 (devel/rubygem-defaultvaluefor32) : 3.2.0 rubygem-ffi110 (devel/rubygem-ffi110) : 1.10.0 rubygem-zeitwerk (devel/rubygem-zeitwerk) : 2.1.9 sems (net/sems) : 1.7.0.g20190822 skypat (devel/skypat) : 3.1.1 tvm (math/tvm) : 0.4.1440 vavoom (games/vavoom) : 1.33_15 vavoom-extras (games/vavoom-extras) : 1.30_4 Deleted Packages (24) geeqie (graphics/geeqie) : Unknown reason iriverter (multimedia/iriverter) : Unknown reason kde5 (x11/kde5) : Unknown reason kicad-doc (cad/kicad-doc) : Unknown reason os-nozfs-buildworld (os/buildworld) : Unknown reason os-nozfs-userland (os/userland) : Unknown reason os-nozfs-userland-base (os/userland-base) : Unknown reason os-nozfs-userland-base-bootstrap (os/userland-base-bootstrap) : Unknown reason os-nozfs-userland-bin (os/userland-bin) : Unknown reason os-nozfs-userland-boot (os/userland-boot) : Unknown reason os-nozfs-userland-conf (os/userland-conf) : Unknown reason os-nozfs-userland-debug (os/userland-debug) : Unknown reason os-nozfs-userland-devtools (os/userland-devtools) : Unknown reason os-nozfs-userland-docs (os/userland-docs) : Unknown reason os-nozfs-userland-lib (os/userland-lib) : Unknown reason os-nozfs-userland-lib32 (os/userland-lib32) : Unknown reason os-nozfs-userland-lib32-development (os/userland-lib32-development) : Unknown reason os-nozfs-userland-rescue (os/userland-rescue) : Unknown reason os-nozfs-userland-sbin (os/userland-sbin) : Unknown reason os-nozfs-userland-tests (os/userland-tests) : Unknown reason photoprint (print/photoprint) : Unknown reason plasma5-plasma (x11/plasma5-plasma) : Unknown reason polkit-qt5 (sysutils/polkit-qt) : Unknown reason secpanel (security/secpanel) : Unknown reason Beastie Bits DragonFlyBSD - msdosfs updates (https://www.dragonflydigest.com/2019/09/10/23472.html) Stand out as a speaker (https://science.sciencemag.org/content/365/6455/834.full) Not a review of the 7th Gen X1 Carbon (http://akpoff.com/archive/2019/not_a_review_of_the_lenovo_x1c7.html) FreeBSD Meets Linux At The Open Source Summit (https://www.tfir.io/2019/08/24/freebsd-meets-linux-at-the-open-source-summit/) QEMU VM Escape (https://blog.bi0s.in/2019/08/24/Pwn/VM-Escape/2019-07-29-qemu-vm-escape-cve-2019-14378/) Porting wine to amd64 on NetBSD, third evaluation report. (http://blog.netbsd.org/tnf/entry/porting_wine_to_amd64_on1) OpenBSD disabled DoH by default in Firefox (https://undeadly.org/cgi?action=article;sid=20190911113856) Feedback/Questions Reinis - GELI with UEFI (http://dpaste.com/0SG8630#wrap) Mason - Beeping (http://dpaste.com/1FQN173) [CHVT feedback] DJ - Feedback (http://dpaste.com/08M3XNH#wrap) Ben - chvt (http://dpaste.com/274RVCE#wrap) Harri - Marc's chvt question (http://dpaste.com/23R1YMK#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 18 Sep 2019 20:00:00 -0700
315: Recapping vBSDcon 2019
vBSDcon 2019 recap, Unix at 50, OpenBSD on fan-less Tuxedo InfinityBook, humungus - an hg server, how to configure a network dump in FreeBSD, and more. Headlines vBSDcon Recap Allan and Benedict attended vBSDcon 2019, which ended last week. It was held again at the Hyatt Regency Reston and the main conference was organized by Dan Langille of BSDCan fame.The two day conference was preceded by a one day FreeBSD hackathon, where FreeBSD developers had the chance to work on patches and PRs. In the evening, a reception was held to welcome attendees and give them a chance to chat and get to know each other over food and drinks. The first day of the conference was opened with a Keynote by Paul Vixie about DNS over HTTPS (DoH). He explained how we got to the current state and what challenges (technical and social) this entails. If you missed this talk and are dying to see it, it will also be presented at EuroBSDCon next week John Baldwin followed up by giving an overview of the work on “In-Kernel TLS Framing and Encryption for FreeBSD” abstract (https://www.vbsdcon.com/schedule/2019-09-06.html#talk:132615) and the recent commit we covered in episode 313. Meanwhile, Brian Callahan was giving a separate session in another room about “Learning to (Open)BSD through its porting system: an attendee-driven educational session” where people had the chance to learn about how to create ports for the BSDs. David Fullard’s talk about “Transitioning from FreeNAS to FreeBSD” was his first talk at a BSD conference and described how he built his own home NAS setup trying to replicate FreeNAS’ functionality on FreeBSD, and why he transitioned from using an appliance to using vanilla FreeBSD. Shawn Webb followed with his overview talk about the “State of the Hardened Union”. Benedict’s talk about “Replacing an Oracle Server with FreeBSD, OpenZFS, and PostgreSQL” was well received as people are interested in how we liberated ourselves from the clutches of Oracle without compromising functionality. Entertaining and educational at the same time, Michael W. Lucas talk about “Twenty Years in Jail: FreeBSD Jails, Then and Now” closed the first day. Lucas also had a table in the hallway with his various tech and non-tech books for sale. People formed small groups and went into town for dinner. Some returned later that night to some work in the hacker lounge or talk amongst fellow BSD enthusiasts. Colin Percival was the keynote speaker for the second day and had an in-depth look at “23 years of software side channel attacks”. Allan reprised his “ELI5: ZFS Caching” talk explaining how the ZFS adaptive replacement cache (ARC) work and how it can be tuned for various workloads. “By the numbers: ZFS Performance Results from Six Operating Systems and Their Derivatives” by Michael Dexter followed with his approach to benchmarking OpenZFS on various platforms. Conor Beh was also a new speaker to vBSDcon. His talk was about “FreeBSD at Work: Building Network and Storage Infrastructure with pfSense and FreeNAS”. Two OpenBSD talks closed the talk session: Kurt Mosiejczuk with “Care and Feeding of OpenBSD Porters” and Aaron Poffenberger with “Road Warrior Disaster Recovery: Secure, Synchronized, and Backed-up”. A dinner and reception was enjoyed by the attendees and gave more time to discuss the talks given and other things until late at night. We want to thank the vBSDcon organizers and especially Dan Langille for running such a great conference. We are grateful to Verisign as the main sponsor and The FreeBSD Foundation for sponsoring the tote bags. Thanks to all the speakers and attendees! humungus - an hg server (https://humungus.tedunangst.com/r/humungus) Features View changes, files, changesets, etc. Some syntax highlighting. Read only. Serves multiple repositories. Allows cloning via the obvious URL. Supports go get. Serves files for downloads. Online documentation via mandoc. Terminal based admin interface. News Roundup OpenBSD on fan-less Tuxedo InfinityBook 14″ v2. (https://hazardous.org/archive/blog/openbsd/2019/09/02/OpenBSD-on-Infinitybook14) The InfinityBook 14” v2 is a fanless 14” notebook. It is an excellent choice for running OpenBSD - but order it with the supported wireless card (see below.). I’ve set it up in a dual-boot configuration so that I can switch between Linux and OpenBSD - mainly to spot differences in the drivers. TUXEDO allows a variety of configurations through their webshop. The dual boot setup with grub2 and EFI boot will be covered in a separate blogpost. My tests were done with OpenBSD-current - which is as of writing flagged as 6.6-beta. See Article for breakdown of CPU, Wireless, Video, Webcam, Audio, ACPI, Battery, Touchpad, and MicroSD Card Reader Unix at 50: How the OS that powered smartphones started from failure (https://arstechnica.com/gadgets/2019/08/unix-at-50-it-starts-with-a-mainframe-a-gator-and-three-dedicated-researchers/) Maybe its pervasiveness has long obscured its origins. But Unix, the operating system that in one derivative or another powers nearly all smartphones sold worldwide, was born 50 years ago from the failure of an ambitious project that involved titans like Bell Labs, GE, and MIT. Largely the brainchild of a few programmers at Bell Labs, the unlikely story of Unix begins with a meeting on the top floor of an otherwise unremarkable annex at the sprawling Bell Labs complex in Murray Hill, New Jersey. It was a bright, cold Monday, the last day of March 1969, and the computer sciences department was hosting distinguished guests: Bill Baker, a Bell Labs vice president, and Ed David, the director of research. Baker was about to pull the plug on Multics (a condensed form of MULTiplexed Information and Computing Service), a software project that the computer sciences department had been working on for four years. Multics was two years overdue, way over budget, and functional only in the loosest possible understanding of the term. Trying to put the best spin possible on what was clearly an abject failure, Baker gave a speech in which he claimed that Bell Labs had accomplished everything it was trying to accomplish in Multics and that they no longer needed to work on the project. As Berk Tague, a staffer present at the meeting, later told Princeton University, “Like Vietnam, he declared victory and got out of Multics.” Within the department, this announcement was hardly unexpected. The programmers were acutely aware of the various issues with both the scope of the project and the computer they had been asked to build it for. Still, it was something to work on, and as long as Bell Labs was working on Multics, they would also have a $7 million mainframe computer to play around with in their spare time. Dennis Ritchie, one of the programmers working on Multics, later said they all felt some stake in the success of the project, even though they knew the odds of that success were exceedingly remote. Cancellation of Multics meant the end of the only project that the programmers in the Computer science department had to work on—and it also meant the loss of the only computer in the Computer science department. After the GE 645 mainframe was taken apart and hauled off, the computer science department’s resources were reduced to little more than office supplies and a few terminals. Some of Allan’s favourite excerpts: In the early '60s, Bill Ninke, a researcher in acoustics, had demonstrated a rudimentary graphical user interface with a DEC PDP-7 minicomputer. Acoustics still had that computer, but they weren’t using it and had stuck it somewhere out of the way up on the sixth floor. And so Thompson, an indefatigable explorer of the labs’ nooks and crannies, finally found that PDP-7 shortly after Davis and Baker cancelled Multics. With the rest of the team’s help, Thompson bundled up the various pieces of the PDP-7—a machine about the size of a refrigerator, not counting the terminal—moved it into a closet assigned to the acoustics department, and got it up and running. One way or another, they convinced acoustics to provide space for the computer and also to pay for the not infrequent repairs to it out of that department’s budget. McIlroy’s programmers suddenly had a computer, kind of. So during the summer of 1969, Thompson, Ritchie, and Canaday hashed out the basics of a file manager that would run on the PDP-7. This was no simple task. Batch computing—running programs one after the other—rarely required that a computer be able to permanently store information, and many mainframes did not have any permanent storage device (whether a tape or a hard disk) attached to them. But the time-sharing environment that these programmers had fallen in love with required attached storage. And with multiple users connected to the same computer at the same time, the file manager had to be written well enough to keep one user’s files from being written over another user’s. When a file was read, the output from that file had to be sent to the user that was opening it. It was a challenge that McIlroy’s team was willing to accept. They had seen the future of computing and wanted to explore it. They knew that Multics was a dead-end, but they had discovered the possibilities opened up by shared development, shared access, and real-time computing. Twenty years later, Ritchie characterized it for Princeton as such: “What we wanted to preserve was not just a good environment in which to do programming, but a system around which a fellowship could form.” Eventually when they had the file management system more or less fleshed out conceptually, it came time to actually write the code. The trio—all of whom had terrible handwriting—decided to use the Labs’ dictating service. One of them called up a lab extension and dictated the entire code base into a tape recorder. And thus, some unidentified clerical worker or workers soon had the unenviable task of trying to convert that into a typewritten document. Of course, it was done imperfectly. Among various errors, “inode” came back as “eye node,” but the output was still viewed as a decided improvement over their assorted scribbles. In August 1969, Thompson’s wife and son went on a three-week vacation to see her family out in Berkeley, and Thompson decided to spend that time writing an assembler, a file editor, and a kernel to manage the PDP-7 processor. This would turn the group’s file manager into a full-fledged operating system. He generously allocated himself one week for each task. Thompson finished his tasks more or less on schedule. And by September, the computer science department at Bell Labs had an operating system running on a PDP-7—and it wasn’t Multics. By the summer of 1970, the team had attached a tape drive to the PDP-7, and their blossoming OS also had a growing selection of tools for programmers (several of which persist down to this day). But despite the successes, Thompson, Canaday, and Ritchie were still being rebuffed by labs management in their efforts to get a brand-new computer. It wasn’t until late 1971 that the computer science department got a truly modern computer. The Unix team had developed several tools designed to automatically format text files for printing over the past year or so. They had done so to simplify the production of documentation for their pet project, but their tools had escaped and were being used by several researchers elsewhere on the top floor. At the same time, the legal department was prepared to spend a fortune on a mainframe program called “AstroText.” Catching wind of this, the Unix crew realized that they could, with only a little effort, upgrade the tools they had written for their own use into something that the legal department could use to prepare patent applications. The computer science department pitched lab management on the purchase of a DEC PDP-11 for document production purposes, and Max Mathews offered to pay for the machine out of the acoustics department budget. Finally, management gave in and purchased a computer for the Unix team to play with. Eventually, word leaked out about this operating system, and businesses and institutions with PDP-11s began contacting Bell Labs about their new operating system. The Labs made it available for free—requesting only the cost of postage and media from anyone who wanted a copy. The rest has quite literally made tech history. See the link for the rest of the article How to configure a network dump in FreeBSD? (https://www.oshogbo.vexillium.org/blog/68/) A network dump might be very useful for collecting kernel crash dumps from embedded machines and machines with a larger amount of RAM then available swap partition size. Besides net dumps we can also try to compress the core dump. However, often this may still not be enough swap to keep whole core dump. In such situation using network dump is a convenient and reliable way for collecting kernel dump. So, first, let’s talk a little bit about history. The first implementation of the network dumps was implemented around 2000 for the FreeBSD 4.x as a kernel module. The code was implemented in 2010 with the intention of being part of FreeBSD 9.0. However, the code never landed in FreeBSD. Finally, in 2018 with the commit r333283 by Mark Johnston the netdump client code landed in the FreeBSD. Subsequently, many other commitments were then implemented to add support for the different drivers (for example r333289). The first official release of FreeBSD, which support netdump is FreeBSD 12.0. Now, let’s get back to the main topic. How to configure the network dump? Two machines are needed. One machine is to collect core dump, let’s call it server. We will use the second one to send us the core dump - the client. See the link for the rest of the article Beastie Bits Sudo Mastery 2nd edition is not out (https://mwl.io/archives/4530) Empirical Notes on the Interaction Between Continuous Kernel Fuzzing and Development (http://users.utu.fi/kakrind/publications/19/vulnfuzz_camera.pdf) soso (https://github.com/ozkl/soso) GregKH - OpenBSD was right (https://youtu.be/gUqcMs0svNU?t=254) Game of Trees (https://gameoftrees.org/faq.html) Feedback/Questions BostJan - Another Question (http://dpaste.com/1ZPCCQY#wrap) Tom - PF (http://dpaste.com/3ZSCB8N#wrap) JohnnyK - Changing VT without keys (http://dpaste.com/3QZQ7Q5#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 11 Sep 2019 22:45:00 -0700
314: Swap that Space
Unix virtual memory when you have no swap space, Dsynth details on Dragonfly, Instant Workstation on FreeBSD, new servers new tech, Experimenting with streaming setups on NetBSD, NetBSD’s progress towards Steam support thanks to GSoC, and more. Headlines What has to happen with Unix virtual memory when you have no swap space (https://utcc.utoronto.ca/~cks/space/blog/unix/NoSwapConsequence) Recently, Artem S. Tashkinov wrote on the Linux kernel mailing list about a Linux problem under memory pressure (via, and threaded here). The specific reproduction instructions involved having low RAM, turning off swap space, and then putting the system under load, and when that happened (emphasis mine): Once you hit a situation when opening a new tab requires more RAM than is currently available, the system will stall hard. You will barely be able to move the mouse pointer. Your disk LED will be flashing incessantly (I'm not entirely sure why). [...] I'm afraid I have bad news for the people snickering at Linux here; if you're running without swap space, you can probably get any Unix to behave this way under memory pressure. If you can't on your particular Unix, I'd actually say that your Unix is probably not letting you get full use out of your RAM. To simplify a bit, we can divide pages of user memory up into anonymous pages and file-backed pages. File-backed pages are what they sound like; they come from some specific file on the filesystem that they can be written out to (if they're dirty) or read back in from. Anonymous pages are not backed by a file, so the only place they can be written out to and read back in from is swap space. Anonymous pages mostly come from dynamic memory allocations and from modifying the program's global variables and data; file backed pages come mostly from mapping files into memory with mmap() and also, crucially, from the code and read-only data of the program. See link for the rest of the article Dsynth details on Dragonfly (https://www.dragonflydigest.com/2019/08/27/23398.html) First, history: DragonFly has had binaries of dports available for download for quite some time. These were originally built using poudriere, and then using the synth tool put together by John Marino. Synth worked both to build all software in dports, and as a way to test DragonFly’s SMP capability under extreme load. Matthew Dillon is working on a new version, called dsynth. It is available now but not yet part of the build. He’s been working quickly on it and there’s plenty more commits than what I have linked here. It’s already led to finding more high-load fixes. dsynth DSynth is basically synth written in C, from scratch. It is designed to give us a bulk builder in base and be friendly to porting and jails down the line (for now its uses chroot's). The original synth was written by John R. Marino and its basic flow was used in writing this program, but as it was written in ada no code was directly copied. The intent is to make dsynth compatible with synth's configuration files and directory structure. This is a work in progress and not yet ready for prime-time. Pushing so we can get some more eyeballs. Most of the directives do not yet work (everything, and build works, and 'cleanup' can be used to clean up any dangling mounts). dsynth code (https://gitweb.dragonflybsd.org/dragonfly.git/blob/HEAD:/usr.bin/dsynth/dsynth.1) News Roundup Instant Workstation (https://euroquis.nl/freebsd/2019/08/12/instant-workstation.html) Some considerable time ago I wrote up instructions on how to set up a FreeBSD machine with the latest KDE Plasma Desktop. Those instructions, while fairly short (set up X, install the KDE meta-port, .. and that’s it) are a bit fiddly. So – prompted slightly by a Twitter exchange recently – I’ve started a mini-sub-project to script the installation of a desktop environment and the bits needed to support it. To give it at least a modicum of UI, dialog(1) is used to ask for an environment to install and a display manager. The tricky bits – pointed out to me after I started – are hardware support, although a best-effort is better than having nothing, I think. In any case, in a VBox host it’s now down to running a single script and picking Plasma and SDDM to get a usable system for me. Other combinations have not been tested, nor has system-hardware-setup. I’ll probably maintain it for a while and if I have time and energy it’ll be tried with nVidia (those work quite well on FreeBSD) and AMD (not so much, in my experience) graphics cards when I shuffle some machines around. Here is the script in my GitHub repository with notes-for-myself. (https://raw.githubusercontent.com/adriaandegroot/FreeBSDTools/master/bin/instant-workstation) New Servers, new Tech (https://www.dragonflydigest.com/2019/08/26/23396.html) Following up on an earlier post, the new servers for DragonFly are in place. The old 40-core machine used for bulk build, monster, is being retired. The power efficiency of the new machines is startling. Incidentally, this is where donations go – infrastructure. New servers in the colo, monster is being retired (http://lists.dragonflybsd.org/pipermail/users/2019-August/358271.html) We have three new servers in the colo now that will be taking most/all bulk package building duties from monster and the two blades (muscles and pkgbox64) that previously did the work. Monster will be retired. The new servers are a dual-socket Xeon (sting) and two 3900X based systems (thor and loki) which all together burn only around half the wattage that monster burned (500W vs 1000W) and 3 times the performance. That's at least a 6:1 improvement in performance efficiency. With SSD prices down significantly the new machines have all-SSDs. These new machines allow us to build dports binary packages for release, master, and staged at the same time and reduces the full-on bulk build times for getting all three done down from 2 weeks to 2 days. It will allow us to more promptly synchronize updates to ports with dports and get binary packages up sooner. Monster, our venerable 48-core quad-socket opteron is being retired. This was a wonderful dev machine for working on DragonFly's SMP algorithms over the last 6+ years precisely because its inter-core and inter-socket latencies were quite high. If a SMP algorithm wasn't spot-on, you could feel it. Over the years DragonFly's performance on monster in doing things like bulk builds increased radically as the SMP algorithms got better and the cores became more and more localized. This kept monster relevant far longer than I thought it would be. But we are at a point now where improvements in efficiency are just too good to ignore. Monster's quad-socket opteron (4 x 12 core 6168's) pulls 1000W under full load while a single Ryzen 3900X (12 core / 24 thread) in a server configuration pulls only 150W, and is slightly faster on the same workload to boot. I would like to thank everyone's generous donations over the last few years! We burned a few thousand on the new machines (as well as the major SSD upgrades we did to the blades) and made very good use of the money, particularly this year as prices for all major components (RAM, SSDs, CPUs, Mobos, etc) have dropped significantly. Experimenting with streaming setups on NetBSD (https://dressupgeekout.blogspot.com/2019/08/experimenting-with-streaming-setups-on.html?m=1) Ever since OBS was successfully ported to NetBSD, I’ve been trying it out, seeing what works and what doesn’t. I’ve only just gotten started, and there’ll definitely be a lot of tweaking going forward. Capturing a specific application’s windows seems to work okay. Capturing an entire display works, too. I actually haven’t tried streaming to Twitch or YouTube yet, but in a previous experiment a few weeks ago, I was able to run a FFmpeg command line and that could stream to Twitch mostly OK. My laptop combined with my external monitor allows me to have a dual-monitor setup wherein the smaller laptop screen can be my “broadcasting station” while the bigger screen is where all the action takes place. I can make OBS visible on all Xfce workspaces, but keep it tucked away on that display only. Altogether, the setup should let me use the big screen for the fun stuff but I can still monitor everything in the small screen. NetBSD Made Progress Thanks To GSoC In Its March Towards Steam Support (https://www.phoronix.com/scan.php?page=news_item&px=NetBSD-Linux-DRM-Ioctl-GSoC2019) Ultimately the goal is to get Valve's Steam client running on NetBSD using their Linux compatibility layer while the focus the past few months with Google Summer of Code 2019 were supporting the necessary DRM ioctls for allowing Linux software running on NetBSD to be able to tap accelerated graphics support. Student developer Surya P spent the summer working on compat_netbsd32 DRM interfaces to allow Direct Rendering Manager using applications running under their Linux compatibility layer. These interfaces have been tested and working as well as updating the "suse131" packages in NetBSD to make use of those interfaces. So the necessary interfaces are now in place for Linux software running on NetBSD to be able to use accelerated graphics though Steam itself isn't yet running on NetBSD with this layer. Those curious about this DRM ioctl GSoC project can learn more from the NetBSD blog (https://blog.netbsd.org/tnf/entry/gsoc_2019_report_implementation_of). NetBSD has also been seeing work this summer on Wayland support and better Wine support to ultimately make this BSD a better desktop operating system and potentially a comparable gaming platform to Linux. Beastie Bits FreeBSD in Wellington? (https://twitter.com/MengTangmu/status/1163265206660694016) FreeBSD on GFE (https://twitter.com/onewilshire/status/1163792878642114560) Clarification (https://twitter.com/onewilshire/status/1166323112620826624) Distrotest.net now with BSDs (https://distrotest.net/) Lecture: Anykernels meet fuzzing NetBSD (https://fahrplan.events.ccc.de/camp/2019/Fahrplan/events/10334.html) Sun Microsystems business plan from 1982 [pdf] (https://www.khoslaventures.com/wp-content/uploads/SunMicrosystem_bus_plan.pdf) Feedback/Questions Alan - Questions (http://dpaste.com/1Z8EGTW) Rodriguez - Feedback and a question (http://dpaste.com/2PZFP4X#wrap) Jeff - OpenZFS follow-up, FreeBSD Adventures (http://dpaste.com/02ZM6YE#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 04 Sep 2019 17:00:00 -0700
313: In-Kernel TLS
OpenBSD on 7th gen Thinkpad X1 Carbon, how to install FreeBSD on a MacBook, Kernel portion of in-kernel TLS (KTLS), Boot Environments on DragonflyBSD, Project Trident Updates, vBSDcon schedule, and more. Headlines OpenBSD on the Thinkpad X1 Carbon 7th Gen (https://jcs.org/2019/08/14/x1c7) Another year, another ThinkPad X1 Carbon, this time with a Dolby Atmos sound system and a smaller battery. The seventh generation X1 Carbon isn't much different than the fifth and sixth generations. I opted for the non-vPro Core i5-8265U, 16Gb of RAM, a 512Gb NVMe SSD, and a matte non-touch WQHD display at ~300 nits. A brighter 500-nit 4k display is available, though early reports indicated it severely impacts battery life. Gone are the microSD card slot on the back and 1mm of overall thickness (from 15.95mm to 14.95mm), but also 6Whr of battery (down to 51Whr) and a little bit of travel in the keyboard and TrackPoint buttons. I still very much like the feel of both of them, so kudos to Lenovo for not going too far down the Apple route of sacrificing performance and usability just for a thinner profile. On my fifth generation X1 Carbon, I used a vinyl plotter to cut out stickers to cover the webcam, "X1 Carbon" branding from the bottom of the display, the power button LED, and the "ThinkPad" branding from the lower part of the keyboard deck. See link for the rest of the article How To Install FreeBSD On A MacBook 1,1 or 2,1 (http://lexploit.com/freebsdmacbook1-1-2-1/) FreeBSD Setup For MacBook 1,1 and 2,1 FreeBSD with some additional setup can be installed on a MacBook 1,1 or 2,1. This article covers how to do so with FreeBSD 10-12. Installing FreeBSD can be installed as the only OS on your MacBook if desired. What you should have is: A Mac OS X 10.4.6-10.7.5 installer. Unofficial versions modified for these MacBooks such as 10.8 also work. A blank CD or DVD to burn the FreeBSD image to. Discs simply work best with these older MacBooks. An ISO file of FreeBSD for x86. The AMD64 ISO does not boot due to the 32 bit EFI of these MacBooks. Burn the ISO file to the blank CD or DVD. Once done, make sure it's in your MacBook and then power off the MacBook. Turn it on, and hold down the c key until the FreeBSD disc boots. See link for the rest of the guide News Roundup Patch for review: Kernel portion of in-kernel TLS (KTLS) (https://svnweb.freebsd.org/base?view=revision&revision=351522) One of the projects I have been working on for the past several months in conjunction with several other folks is upstreaming work from Netflix to handle some aspects of Transport Layer Security (TLS) in the kernel. In particular, this lets a web server use sendfile() to send static content on HTTPS connections. There is a lot more detail in the review itself, so I will spare pasting a big wall of text here. However, I have posted the patch to add the kernel-side of KTLS for review at the URL below. KTLS also requires other patches to OpenSSL and nginx, but this review is only for the kernel bits. Patches and reviews for the other bits will follow later. https://reviews.freebsd.org/D21277 DragonFly Boot Enviroments (https://github.com/newnix/dfbeadm) This is a tool inspired by the beadm utility for FreeBSD/Illumos systems that creates and manages ZFS boot environments. This utility in contrast is written from the ground up in C, this should provide better performance, integration, and extensibility than the POSIX sh and awk script it was inspired by. During the time this project has been worked on, beadm has been superseded by bectl on FreeBSD. After hammering out some of the outstanding internal logic issues, I might look at providing a similar interface to the command as bectl. See link for the rest of the details Project Trident Updates 19.08 Available (https://project-trident.org/post/2019-08-15_19.08_available/) This is a general package update to the CURRENT release repository based upon TrueOS 19.08. Legacy boot ISO functional again This update includes the FreeBSD fixes for the “vesa” graphics driver for legacy-boot systems. The system can once again be installed on legacy-boot systems. PACKAGE CHANGES FROM 19.07-U1 New Packages: 154 Deleted Packages: 394 Updated Packages: 4926 12-U3 Available (https://project-trident.org/post/2019-08-22_stable12-u3_available/) This is the third general package update to the STABLE release repository based upon TrueOS 12-Stable. PACKAGE CHANGES FROM STABLE 12-U2 New Packages: 105 Deleted Packages: 386 Updated Packages: 1046 vBSDcon (https://www.vbsdcon.com/schedule/) vBSDcon 2019 will return to the Hyatt Regency in Reston, VA on September 5-7 2019. *** Beastie Bits The next NYCBUG meeting will be Sept 4 @ 18:45 (https://www.nycbug.org/index?action=view&id=10671) Feedback/Questions Tom - Questions (http://dpaste.com/1AXXK7G#wrap) Michael - dfbeadm (http://dpaste.com/0PNEDYT#wrap) Bostjan - Questions (http://dpaste.com/1N7T7BR#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 28 Aug 2019 21:30:00 -0700
312: Why Package Managers
The UNIX Philosophy in 2019, why use package managers, touchpad interrupted, Porting wine to amd64 on NetBSD second evaluation report, Enhancing Syzkaller Support for NetBSD, all about the Pinebook Pro, killing a process and all of its descendants, fast software the best software, and more. Headlines The UNIX Philosophy in 2019 (https://triosdevelopers.com/jason.eckert/blog/Entries/2019/6/1_Entry_1.html) Today, Linux and open source rules the world, and the UNIX philosophy is widely considered compulsory. Organizations are striving to build small, focused applications that work collaboratively in a cloud and microservices environment. We rely on the network, as well as HTTP (text) APIs for storing and referencing data. Moreover, nearly all configuration is stored and communicated using text (e.g. YAML, JSON or XML). And while the UNIX philosophy has changed dramatically over the past 5 decades, it hasn’t strayed too far from Ken Thompson’s original definition in 1973: We write programs that do one thing and do it well We write programs to work together And we write programs that handle text streams, because that is a universal interface Why Use Package Managers? (https://uwm.edu/hpc/software-management/) Valuable research is often hindered or outright prevented by the inability to install software. This need not be the case. Since I began supporting research computing in 1999, I’ve frequently seen researchers struggle for days or weeks trying to install a single open source application. In most cases, they ultimately failed. In many cases, they could have easily installed the software in seconds with one simple command, using a package manager such as Debian packages, FreeBSD ports, MacPorts, or Pkgsrc, just to name a few. Developer websites often contain poorly written instructions for doing “caveman installs”; manually downloading, unpacking, patching, and building the software. The same laborious process must often be followed for other software packages on which it depends, which can sometimes number in the dozens. Many researchers are simply unaware that there are easier ways to install the software they need. Caveman installs are a colossal waste of man-hours. If 1000 people around the globe spend an average of 20 hours each trying to install the same program that could have been installed with a package manager (this is not uncommon), then 20,000 man-hours have been lost that could have gone toward science. How many important discoveries are delayed by this? The elite research institutions have ample funding and dozens of IT staff dedicated to research computing. They can churn out publications even if their operation is inefficient. Most institutions, however, have few or no IT staff dedicated to research, and cannot afford to squander precious man-hours on temporary, one-off software installs. The wise approach for those of us in that situation is to collaborate on making software deployment easier for everyone. If we do so, then even the smallest research groups can leverage that work to be more productive and make more frequent contributions to science. Fortunately, the vast majority of open source software installs can be made trivial for anyone to do for themselves. Modern package managers perform all the same steps as a caveman install, but automatically. Package managers also install dependencies for us automatically. News Roundup Touchpad, Interrupted (https://jcs.org/2019/07/28/ihidev) For two years I've been driving myself crazy trying to figure out the source of a driver problem on OpenBSD: interrupts never arrived for certain touchpad devices. A couple weeks ago, I put out a public plea asking for help in case any non-OpenBSD developers recognized the problem, but while debugging an unrelated issue over the weekend, I finally solved it. It's been a long journey and it's a technical tale, but here it is. Porting wine to amd64 on NetBSD, second evaluation report (https://blog.netbsd.org/tnf/entry/porting_wine_to_amd64_on2) Summary Presently, Wine on amd64 is in test phase. It seems to work fine with caveats like LDLIBRARYPATH which has to be set as 32-bit Xorg libs don't have ${PREFIX}/emul/netbsd32/lib in its rpath section. The latter is due to us extracting 32-bit libs from tarballs in lieu of building 32-bit Xorg on amd64. As previously stated, pkgsrc doesn't search for pkgconfig files in ${PREFIX}/emul/netbsd32/lib which might have inadvertent effects that I am unaware of as of now. I shall be working on these issues during the final coding period. I would like to thank @leot, @maya and @christos for saving me from shooting myself in the foot many a time. I, admittedly, have had times when multiple approaches, which all seemed right at that time, perplexed me. I believe those are times when having a mentor counts, and I have been lucky enough to have really good ones. Once again, thanks to Google for this wonderful opportunity. Enhancing Syzkaller Support for NetBSD, Part 2 (https://blog.netbsd.org/tnf/entry/enchancing_syzkaller_support_for_netbsd) As a part of Google Summer of Code’19, I am working on improving the support for Syzkaller kernel fuzzer. Syzkaller is an unsupervised coverage-guided kernel fuzzer, that supports a variety of operating systems including NetBSD. This report details the work done during the second coding period. You can also take a look at the first report to learn more about the initial support that we added. : https://blog.netbsd.org/tnf/entry/enhancingsyzkallersupportfornetbsd July Update: All about the Pinebook Pro (https://www.pine64.org/2019/07/05/july-update-all-about-the-pinebook-pro/) "So I said I won’t be talking about the BSDs, but I feel like I should at the very least give you a general overview of the RK3399 *BSD functionality. I’ll make it quick. I’ve spoken to *BSD devs whom worked on the RockPro64 and from what I’ve gathered (despite the different *BSDs having varying degree of support for the RK3399 SOC) many of the core features are already supported, which bodes well for *BSD on the Pro. That said, some of the things you’d require on a functional laptop – such as the LCD (using eDP) for instance – will not work on the Pinebook Pro using *BSD as of today. So clearly a degree of work is yet needed for a BSD to run on the device. However, keep in mind that *BSD developers will be receiving their units soon and by the time you receive yours some basic functionality may be available." Killing a process and all of its descendants (http://morningcoffee.io/killing-a-process-and-all-of-its-descendants.html) Killing processes in a Unix-like system can be trickier than expected. Last week I was debugging an odd issue related to job stopping on Semaphore. More specifically, an issue related to the killing of a running process in a job. Here are the highlights of what I learned: Unix-like operating systems have sophisticated process relationships. Parent-child, process groups, sessions, and session leaders. However, the details are not uniform across operating systems like Linux and macOS. POSIX compliant operating systems support sending signals to process groups with a negative PID number. Sending signals to all processes in a session is not trivial with syscalls. Child processes started with exec inherit their parent signal configuration. If the parent process is ignoring the SIGHUP signal, for example, this configuration is propagated to the children. The answer to the “What happens with orphaned process groups” question is not trivial. Fast Software, the Best Software (https://craigmod.com/essays/fast_software/) I love fast software. That is, software speedy both in function and interface. Software with minimal to no lag between wanting to activate or manipulate something and the thing happening. Lightness. Software that’s speedy usually means it’s focused. Like a good tool, it often means that it’s simple, but that’s not necessarily true. Speed in software is probably the most valuable, least valued asset. To me, speedy software is the difference between an application smoothly integrating into your life, and one called upon with great reluctance. Fastness in software is like great margins in a book — makes you smile without necessarily knowing why. But why is slow bad? Fast software is not always good software, but slow software is rarely able to rise to greatness. Fast software gives the user a chance to “meld” with its toolset. That is, not break flow. When the nerds upon Nerd Hill fight to the death over Vi and Emacs, it’s partly because they have such a strong affinity for the flow of the application and its meldiness. They have invested. The Tool Is Good, so they feel. Not breaking flow is an axiom of great tools. A typewriter is an excellent tool because, even though it’s slow in a relative sense, every aspect of the machine itself operates as quickly as the user can move. It is focused. There are no delays when making a new line or slamming a key into the paper. Yes, you have to put a new sheet of paper into the machine at the end of a page, but that action becomes part of the flow of using the machine, and the accumulation of paper a visual indication of work completed. It is not wasted work. There are no fundamental mechanical delays in using the machine. The best software inches ever closer to the physical directness of something like a typewriter. (The machine may break down, of course, ribbons need to be changed — but this is maintenance and separate from the use of the tool. I’d be delighted to “maintain” Photoshop if it would lighten it up.) Beastie Bits Register for vBSDCon 2019, Sept 5-7 in Reston VA (https://vbsdcon.com/registration) Register for EuroBSDCon 2019, Sept 19-22 in Lillehammer, Norway (https://2019.eurobsdcon.org/registration/) Feedback/Questions Paulo - FreeNAS Question (http://dpaste.com/2GDG7WR#wrap) Marc - Changing VT without function keys? (http://dpaste.com/1AKC7A1#wrap) Caleb - Patch, update, and upgrade management (http://dpaste.com/2D6J482#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 21 Aug 2019 20:00:00 -0700
311: Conference Gear Breakdown
NetBSD 9.0 release process has started, xargs, a tale of two spellcheckers, Adapting TriforceAFL for NetBSD, Exploiting a no-name freebsd kernel vulnerability, and more. Headlines NetBSD 9.0 release process has started (https://mail-index.netbsd.org/netbsd-announce/2019/07/31/msg000301.html) If you have been following source-changes, you may have noticed the creation of the netbsd-9 branch! It has some really exciting items that we worked on: + New AArch64 architecture support: + Symmetric and asymmetrical multiprocessing support (aka big.LITTLE) + Support for running 32-bit binaries + UEFI and ACPI support + Support for SBSA/SBBR (server-class) hardware. + The FDT-ization of many ARM boards: + the 32-bit GENERIC kernel lists 129 different DTS configurations + the 64-bit GENERIC64 kernel lists 74 different DTS configurations + All supported by a single kernel, without requiring per-board configuration. + Graphics driver update, matching Linux 4.4, adding support for up to Kaby Lake based Intel graphics devices. + ZFS has been updated to a modern version and seen many bugfixes. + New hardware-accelerated virtualization via NVMM. + NPF performance improvements and bug fixes. A new lookup algorithm, thmap, is now the default. + NVMe performance improvements + Optional kernel ASLR support, and partial kernel ASLR for the default configuration. + Kernel sanitizers: + KLEAK, detecting memory leaks + KASAN, detecting memory overruns + KUBSAN, detecting undefined behaviour + These have been used together with continuous fuzzing via the syzkaller project to find many bugs that were fixed. + The removal of outdated networking components such as ISDN and all of its drivers + The installer is now capable of performing GPT UEFI installations. + Dramatically improved support for userland sanitizers, as well as the option to build all of NetBSD's userland using them for bug-finding. + Update to graphics userland: Mesa was updated to 18.3.4, and llvmpipe is now available for several architectures, providing 3D graphics even in the absence of a supported GPU. We try to test NetBSD as best as we can, but your testing can help NetBSD 9.0 a great release. Please test it and let us know of any bugs you find. + Binaries are available at https://nycdn.netbsd.org/pub/NetBSD-daily/netbsd-9/latest/ xargs wtf (https://medium.com/@aarontharris/xargs-wtf-34d2618286b7) xargs is probably one of the more difficult to understand of the unix command arsenal and of course that just means it’s one of the most useful too. I discovered a handy trick that I thought was worth a share. Please note there are probably other (better) ways to do this but I did my stackoverflow research and found nothing better. xargs — at least how I’ve most utilized it — is handy for taking some number of lines as input and doing some work per line. It’s hard to be more specific than that as it does so much else. It literally took me an hour of piecing together random man pages + tips from 11 year olds on stack overflow, but eventually I produced this gem: This is an example of how to find files matching a certain pattern and rename each of them. It sounds so trivial (and it is) but it demonstrates some cool tricks in an easy concept. News Roundup PkgSrc: A Tale of Two Spellcheckers (https://bentsukun.ch/posts/pkgsrccon-2019/) This is a transcript of the talk I gave at pkgsrcCon 2019 in Cambridge, UK. It is about spellcheckers, but there are much more general software engineering lessons that we can learn from this case study. The reason I got into this subject at all was my paternal leave last year, when I finally had some more time to spend working on pkgsrc. It was a tiny item in the enormous TODO file at the top of the source tree (“update enchant to version 2.2”) that made me go into this rabbit hole. Adapting TriforceAFL for NetBSD, Part 2 (https://blog.netbsd.org/tnf/entry/adapting_triforceafl_for_netbsd_part1) I have been working on adapting TriforceAFL for NetBSD kernel syscall fuzzing. This blog post summarizes the work done until the second evaluation. For work done during the first coding period, check out this post. Summary > So far, the TriforceNetBSDSyscallFuzzer has been made available in the form of a pkgsrc package with the ability to fuzz most of NetBSD syscalls. In the final coding period of GSoC. I plan to analyse the crashes that were found until now. Integrate sanitizers, try and find more bugs and finally wrap up neatly with detailed documentation. > Last but not least, I would like to thank my mentor, Kamil Rytarowski for helping me through the process and guiding me. It has been a wonderful learning experience so far! Exploiting a no-name freebsd kernel vulnerability (https://www.synacktiv.com/posts/exploit/exploiting-a-no-name-freebsd-kernel-vulnerability.html) A new patch has been recently shipped in FreeBSD kernels to fix a vulnerability (cve-2019-5602) present in the cdrom device. In this post, we will introduce the bug and discuss its exploitation on pre/post-SMEP FreeBSD revisions. > A closer look at the commit 6bcf6e3 shows that when invoking the CDIOCREADSUBCHANNEL_SYSSPACE ioctl, data are copied with bcopy instead of the copyout primitive. This endows a local attacker belonging to the operator group with an arbitrary write primitive in the kernel memory. [Allan and Benedicts Conference Gear Breakdown] Benedict’s Gear: GlocalMe G3 Mobile Travel HotSpot and Powerbank (https://www.glocalme.com/CA/en-US/cloudsim/g3) Mogics Power Bagel (http://www.mogics.com/3824-2) Charby Sense Power Cable (https://charbycharge.com/charby-sense-worlds-smartest-auto-cutoff-cable/) Allan’s Gear: Huawei E5770s-320 4G LTE 150 Mbps Mobile WiFi Pro (https://smile.amazon.com/gp/product/B013CEGGKI/) AOW Global Data SIM Card for On-Demand 4G LTE Mobile Data in Over 90 Countries (https://smile.amazon.com/dp/B071HJFX27/) All my devices charge from USB-C, so that is great More USB thumb drives than strictly necessary My Lenovo X270 laptop running FreeBSD 13-current My 2016 Macbook Pro (a prize from the raffle at vBSDCon 2017) that I use for email and video conferencing to preserve battery on my FreeBSD machine for work Beastie Bits Replacing the Unix tradition (Warning may be rage inducing) (https://www.youtube.com/watch?v=L9v4Mg8wi4U&feature=youtu.be) Installing OpenBSD over remote serial on the AtomicPI (https://www.thanassis.space/remoteserial.html#remoteserial) Zen 2 and DragonFly (https://www.dragonflydigest.com/2019/08/05/23294.html) Improve Docking on FreeBSD (https://blog.yukiisbo.red/posts/2019/05/improve-docking-on-freebsd/) Register for vBSDCon 2019, Sept 5-7 in Reston VA. Early bird ends August 15th. (https://vbsdcon.com/registration) Register for EuroBSDCon 2019, Sept 19-22 in Lillehammer, Norway (https://2019.eurobsdcon.org/registration/) Feedback/Questions JT - Congrats (http://dpaste.com/0D7Y31E#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 15 Aug 2019 06:00:00 -0700
310: My New Free NAS
OPNsense 19.7.1 is out, ZFS on Linux still has annoying issues with ARC size, Hammer2 is now default, NetBSD audio – an application perspective, new FreeNAS Mini, and more. Headlines OPNsense 19.7.1 (https://opnsense.org/opnsense-19-7-1-released/) We do not wish to keep you from enjoying your summer time, but this is a recommended security update enriched with reliability fixes for the new 19.7 series. Of special note are performance improvements as well as a fix for a longstanding NAT before IPsec limitation. Full patch notes: system: do not create automatic copies of existing gateways system: do not translate empty tunables descriptions system: remove unwanted form action tags system: do not include Syslog-ng in rc.freebsd handler system: fix manual system log stop/start/restart system: scoped IPv6 "%" could confuse mwexecf(), use plain mwexec() instead system: allow curl-based downloads to use both trusted and local authorities system: fix group privilege print and correctly redirect after edit system: use cached address list in referrer check system: fix Syslog-ng search stats firewall: HTML-escape dynamic entries to display aliases firewall: display correct IP version in automatic rules firewall: fix a warning while reading empty outbound rules configuration firewall: skip illegal log lines in live log interfaces: performance improvements for configurations with hundreds of interfaces reporting: performance improvements for Python 3 NetFlow aggregator rewrite dhcp: move advanced router advertisement options to correct config section ipsec: replace global array access with function to ensure side-effect free boot ipsec: change DPD action on start to "dpdaction = restart" ipsec: remove already default "dpdaction = none" if not set ipsec: use interface IP address in local ID when doing NAT before IPsec web proxy: fix database reset for Squid 4 by replacing use of sslcrtd with securityfile_certgen plugins: os-acme-client 1.24[1] plugins: os-bind 1.6[2] plugins: os-dnscrypt-proxy 1.5[3] plugins: os-frr now restricts characters BGP prefix-list and route-maps[4] plugins: os-google-cloud-sdk 1.0[5] ports: curl 7.65.3[6] ports: monit 5.26.0[7] ports: openssh 8.0p1[8] ports: php 7.2.20[9] ports: python 3.7.4[10] ports: sqlite 3.29.0[11] ports: squid 4.8[12] Stay safe and hydrated, Your OPNsense team ZFS on Linux still has annoying issues with ARC size (https://utcc.utoronto.ca/~cks/space/blog/linux/ZFSOnLinuxARCShrinkage) One of the frustrating things about operating ZFS on Linux is that the ARC size is critical but ZFS's auto-tuning of it is opaque and apparently prone to malfunctions, where your ARC will mysteriously shrink drastically and then stick there. Linux's regular filesystem disk cache is very predictable; if you do disk IO, the cache will relentlessly grow to use all of your free memory. This sometimes disconcerts people when free reports that there's very little memory actually free, but at least you're getting value from your RAM. This is so reliable and regular that we generally don't think about 'is my system going to use all of my RAM as a disk cache', because the answer is always 'yes'. (The general filesystem cache is also called the page cache.) This is unfortunately not the case with the ZFS ARC in ZFS on Linux (and it wasn't necessarily the case even on Solaris). ZFS has both a current size and a 'target size' for the ARC (called 'c' in ZFS statistics). When your system boots this target size starts out as the maximum allowed size for the ARC, but various events afterward can cause it to be reduced (which obviously limits the size of your ARC, since that's its purpose). In practice, this reduction in the target size is both pretty sticky and rather mysterious (as ZFS on Linux doesn't currently expose enough statistics to tell why your ARC target size shrunk in any particular case). The net effect is that the ZFS ARC is not infrequently quite shy and hesitant about using memory, in stark contrast to Linux's normal filesystem cache. The default maximum ARC size starts out as only half of your RAM (unlike the regular filesystem cache, which will use all of it), and then it shrinks from there, sometimes very significantly, and once shrunk it only recovers slowly (if at all). News Roundup Hammer2 is now default (http://lists.dragonflybsd.org/pipermail/commits/2019-June/718989.html) ``` commit a49112761c919d42d405ec10252eb0553662c824 Author: Matthew Dillon Date: Mon Jun 10 17:53:46 2019 -0700 installer - Default to HAMMER2 * Change the installer default from HAMMER1 to HAMMER2. * Adjust the nrelease build to print the location of the image files when it finishes. Summary of changes: nrelease/Makefile | 2 +- usr.sbin/installer/dfuibe_installer/flow.c | 20 ++++++++++---------- 2 files changed, 11 insertions(+), 11 deletions(-) http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/a49112761c919d42d405ec10252eb0553662c824 ``` NetBSD audio – an application perspective (https://netbsd.org/gallery/presentations/nia/netbsd-audio/) NetBSD audio – an application perspective ... or, "doing it natively, because we can" audio options for NetBSD in pkgsrc Use NetBSD native audio (sun audio/audioio.h) Or OSS emulation layer: Basically a wrapper around sun audio in the kernel. Incomplete and old version, but works for simple stuff Many many abstraction layers available: OpenAL-Soft alsa-lib (config file required) libao, GStreamer (plugins!) PortAudio, SDL PulseAudio, JACK ... lots more!? some obsolete stuff (esd, nas?) Advantages of using NetBSD audio directly Low latency, low CPU usage: Abstraction layers differ in latency (SDL2 vs ALSA/OpenAL) Query device information: Is /dev/audio1 a USB microphone or another sound card? Avoid bugs from excessive layering Nice API, well documented: [nia note: I had no idea how to write audio code. I read a man page and now I do.] Your code might work on illumos too [nia note: SDL2 seems very sensitive to the blk_ms sysctl being high or low, with other implementations there seems to be a less noticable difference. I don't know why.] New FreeNAS Mini (https://www.ixsystems.com/blog/new-freenas-mini-models-release-pr/) Two new FreeNAS Mini systems join the very popular FreeNAS Mini and Mini XL: FreeNAS Mini XL+: This powerful 10 Bay platform (8x 3.5” and 1x 2.5” hot-swap, 1x 2.5” internal) includes the latest, compact server technology and provides dual 10GbE ports, 8 CPU cores and 32 GB RAM for high performance workgroups. The Mini XL+ scales beyond 100TB and is ideal for very demanding applications, including hosting virtual machines and multimedia editing. Starting at $1499, the Mini XL+ configured with cache SSD and 80 TB capacity is $4299, and consumes about 100 Watts. FreeNAS Mini E: This cost-effective 4 Bay platform provides the resources required for SOHO use with quad GbE ports and 8 GB of RAM. The Mini E is ideal for file sharing, streaming and transcoding video at 1080p. Starting at $749, the Mini E configured with 8 TB capacity is $999, and consumes about 36 Watts. Beastie Bits Welcome to NetBSD 9.99.1! (https://mail-index.netbsd.org/source-changes/2019/07/30/msg107671.html) Berkeley smorgasbord — part II (http://blog.snailtext.com/posts/berkeley-smorgasbord-part-2.html) dtracing postgres (https://www.youtube.com/watch?v=Brt41xnMZqo&list=PLuJmmKtsV1dOTmlImlD9U5j1P1rLxS2V8&index=20&t=0s) Project Trident 19.07-U1 now available (https://project-trident.org/post/2019-07-30_19.07-u1_available/) Need a Secure Operating System? Take a Look at OpenBSD (https://www.devprojournal.com/technology-trends/operating-systems/need-a-secure-operating-system-take-a-look-at-openbsd/) Feedback/Questions Jeff - OpenZFS Port Testing Feedback (http://dpaste.com/2AT7JGP#wrap) Malcolm - Best Practices for Custom Ports (http://dpaste.com/1R170D7) Michael - Little Correction (http://dpaste.com/0CERP6R) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 07 Aug 2019 20:00:00 -0700
Episode 309: Get Your Telnet Fix
DragonFlyBSD Project Update - colo upgrade, future trends, resuming ZFS send, realtime bandwidth terminal graph visualization, fixing telnet fixes, a chapter from the FBI’s history with OpenBSD and an OpenSSH vuln, and more. Headlines DragonFlyBSD Project Update - colo upgrade, future trends (http://lists.dragonflybsd.org/pipermail/users/2019-July/358226.html) For the last week I've been testing out a replacement for Monster, our 48-core opteron server. The project will be removing Monster from the colo in a week or two and replacing it with three machines which together will use half the power that Monster did alone. The goal is to clear out a little power budget in the colo and to really beef-up our package-building capabilities to reduce the turn-around time needed to test ports syncs and updates to the binary package system. Currently we use two blades to do most of the building, plus monster sometimes. The blades take almost a week (120 hours+) to do a full synth run and monster takes around 27.5 hours. But we need to do three bulk builds more or less at the same time... one for the release branch, one for the development branch, and one for staging updates. It just takes too long and its been gnawing at me for a little while. Well, Zen 2 to the rescue! These new CPUs can take ECC, there's actually an IPMI mobo available, and they are fast as hell and cheap for what we get. The new machines will be two 3900X based servers, plus a dual-xeon system that I already had at home. The 3900X's can each do a full synth run in 24.5 hours and the Xeon can do it in around 31 hours. Monster will be retired. And the crazy thing about this? Monster burns 1000W going full bore. Each of the 3900X servers burns 160W and the Xeon burns 200W. In otherwords, we are replacing 1000W with only 520W and getting roughly 6x the performance efficiency in the upgrade. This tell you just how much more power-efficient machines have become in the last 9 years or so. > This upgrade will allow us to do full builds for both release and dev in roughly one day instead of seven days, and do it without interfering with staging work that might be happening at the same time. Future trends - DragonFlyBSD has reached a bit of a cross-roads. With most of the SMP work now essentially complete across the entire system the main project focus is now on supplying reliable binary ports for release and developer branches, DRM (GPU) support and other UI elements to keep DragonFlyBSD relevant on workstations, and continuing Filesystem work on HAMMER2 to get multi-device and clustering going. Resuming ZFS send (https://www.oshogbo.vexillium.org/blog/66/) One of the amazing functionalities of ZFS is the possibility of sending a whole dataset from one place to another. This mechanism is amazing to create backups of your ZFS based machines. Although, there were some issues with this functionality for a long time when a user sent a big chunk of data. What if you would do that over the network and your connection has disappeared? What if your machine was rebooted as you are sending a snapshot? For a very long time, you didn't have any options - you had to send a snapshot from the beginning. Now, this limitation was already bad enough. However, another downside of this approach was that all the data which you already send was thrown away. Therefore, ZFS had to go over all this data and remove them from the dataset. Imagine the terabytes of data which you sent via the network was thrown away because as you were sending the last few bytes, the network went off. In this short post, I don't want to go over the whole ZFS snapshot infrastructure (if you think that such a post would be useful, please leave a comment). Now, to get back to the point, this infrastructure is used to clone the datasets. Some time ago a new feature called “Resuming ZFS send” was introduced. That means that if there was some problem with transmitting the dataset from one point to another you could resume it or throw them away. But the point is, that yes, you finally have a choice. News Roundup Realtime bandwidth terminal graph visualization (https://dataswamp.org/~solene/2019-07-19-ttyplot-netstat-openbsd.html) If for some reasons you want to visualize your bandwidth traffic on an interface (in or out) in a terminal with a nice graph, here is a small script to do so, involving ttyplot, a nice software making graphics in a terminal. The following will works on OpenBSD. You can install ttyplot by pkg_add ttyplot as root, ttyplot package appeared since OpenBSD 6.5. fixing telnet fixes (https://flak.tedunangst.com/post/fixing-telnet-fixes) There’s a FreeBSD commit to telnet. fix a couple of snprintf() buffer overflows. It’s received a bit of attention for various reasons, telnet in 2019?, etc. I thought I’d take a look. Here’s a few random observations. The first line is indented with spaces while the others use tabs. The correct type for string length is size_t not unsigned int. sizeof(char) is always one. There’s no need to multiply by it. If you do need to multiply by a size, this is an unsafe pattern. Use calloc or something similar. (OpenBSD provides reallocarray to avoid zeroing cost of calloc.) Return value of malloc doesn’t need to be cast. In fact, should not be, lest you disguise a warning. Return value of malloc is not checked for NULL. No reason to cast cp to char * when passing to snprintf. It already is that type. And if it weren’t, what are you doing? The whole operation could be simplified by using asprintf. Although unlikely (probably impossible here, but more generally), adding the two source lengths together can overflow, resulting in truncation with an unchecked snprintf call. asprintf avoids this failure case. A Chapter from the FBI’s History with OpenBSD and an OpenSSH Vuln (https://twitter.com/RooneyMcNibNug/status/1152327783055601664) Earlier this year I FOIAed the FBI for details on allegations of backdoor installed in the IPSEC stack in 2010, originally discussed by OpenBSD devs (https://marc.info/?l=openbsd-tech&m=129236621626462 …) Today, I got an interesting but unexpected responsive record: Freedom of Information Act: FBI: OpenBSD (https://www.muckrock.com/foi/united-states-of-america-10/foia-fbi-openbsd-70084/) GitHub Repo (https://github.com/RooneyMcNibNug/FOIA/blob/master/Responsive%20Docs/OpenBSD/FBI_OpenBSD_response_OCRd.pdf) Beastie Bits “Sudo Mastery, 2nd Edition” open for tech review (https://mwl.io/archives/4378) FreeBSD Journal: FreeBSD for Makers (https://www.freebsdnews.com/2019/07/12/freebsd-journal-freebsd-for-makers/) OpenBSD and NetBSD machines at Open Source Conference 2019 Nagoya (http://mail-index.netbsd.org/netbsd-advocacy/2019/07/19/msg000808.html) FreeBSD 12.0: WINE Gaming (https://www.youtube.com/watch?v=zuj9pRNR2oM) Introduction to the Structure and Interpretation of TNF (The NetBSD Foundation) (https://www.netbsd.org/gallery/presentations/wiz/pkgsrccon2019/index.html#/) vBSDcon speakers announced (https://www.vbsdcon.com/) Feedback/Questions Pat - NYCBug Aug 7th (http://dpaste.com/21Y1PRM) Tyler - SSH keys vs password (http://dpaste.com/3JEVVEF#wrap) Lars - Tor-Talk (http://dpaste.com/0RAFMXZ) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 31 Jul 2019 20:45:00 -0700
308: Mumbling with OpenBSD
Replacing a (silently) failing disk in a ZFS pool, OPNsense 19.7 RC1 released, implementing DRM ioctl support for NetBSD, High quality/low latency VOIP server with umurmur/Mumble on OpenBSD, the PDP-7 where Unix began, LLDB watchpoints, and more. Headlines Replacing a (silently) failing disk in a ZFS pool (https://imil.net/blog/2019/07/02/Replacing-a-silently-failing-disk-in-a-ZFS-pool/) Maybe I can’t read, but I have the feeling that official documentations explain every single corner case for a given tool, except the one you will actually need. My today’s struggle: replacing a disk within a FreeBSD ZFS pool. What? there’s a shitton of docs on this topic! Are you stupid? I don’t know, maybe. Yet none covered the process in a simple, straight and complete manner. OPNsense 19.7 RC1 released (https://opnsense.org/opnsense-19-7-rc1-released/) Hi there, For four and a half years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing. We thank all of you for helping test, shape and contribute to the project! We know it would not be the same without you. Download links, an installation guide[1] and the checksums for the images can be found below as well. News Roundup Implementation of DRM ioctl Support for NetBSD kernel (https://blog.netbsd.org/tnf/entry/implementation_of_drm_ioctl_support) What is DRM ioctl ? Ioctls are input/output control system calls and DRM stands for direct rendering manager The DRM layer provides several services to graphics drivers, many of them driven by the application interfaces it provides through libdrm, the library that wraps most of the DRM ioctls. These include vblank event handling, memory management, output management, framebuffer management, command submission & fencing, suspend/resume support, and DMA services. Native DRM ioctl calls NetBSD was able to make native DRM ioctl calls with hardware rendering once xorg and proper mesa packages where installed. We used the glxinfo and glxgears applications to test this out. High quality / low latency VOIP server with umurmur/Mumble on OpenBSD (https://dataswamp.org/~solene/2019-07-04-umurmur.html) Discord users keep telling about their so called discord server, which is not dedicated to them at all. And Discord has a very bad quality and a lot of voice distorsion. Why not run your very own mumble server with high voice quality and low latency and privacy respect? This is very easy to setup on OpenBSD! Mumble is an open source voip client, it has a client named Mumble (available on various operating system) and at least Android, the server part is murmur but there is a lightweight server named umurmur. People authentication is done through certificate generated locally and automatically accepted on a server, and the certificate get associated with a nickname. Nobody can pick the same nickname as another person if it’s not the same certificate. TMWL June’19 — JS Fetch API, scheduling in Spring, thoughts on Unix (https://blog.softwaremill.com/tmwl-june19-js-fetch-api-scheduling-in-spring-thoughts-on-unix-fd54f50ecd64) Unix — going back to the roots From time to time, I like to review my knowledge in a certain area, even when I feel like I know a lot about it already. I go back to the basics and read tutorials, manuals, books or watch interesting videos. I’ve been using macOS for a couple of years now, previously being a linux user for some (relatively short) time. Both these operating systems have a common ancestor — Unix. While I’m definitely not an expert, I feel quite comfortable using linux & macOS — I understand the concepts behind the system architecture, know a lot of command line tools & navigate through the shell without a hassle. So-called unix philosophy is also close to my heart. I always feel like there’s more I could squeeze out of it. Recently, I found that book titled “Unix for dummies, 5th edition” which was published back in… 2004. Feels literally like AGES in the computer-related world. However, it was a great shot — the book starts with the basics, providing some brief history of Unix and how it came to life. It talks a lot about the structure of the system and where certain pieces fit (eg. “standard” set of tools), and how to understand permissions and work with files & directories. There’s even a whole chapter about shell-based text editors like Vi and Emacs! Despite the fact that I am familiar with most of these, I could still find some interesting pieces & tools that I either knew existed (but never had a chance to use), or even haven’t ever heard of. And almost all of these are still valid in the modern “incarnations” of Unix’s descendants: Linux and macOS. The book also talks about networking, surfing the web & working with email. It’s cute to see pictures of those old browsers rendering “ancient” Internet websites, but hey — this is how it looked like no more than fifteen years ago! I can really recommend this book to anyone working on modern macOS or Linux — you will certainly find some interesting pieces. Especially if you like to go back to the roots from time to time as I do! ThePDP-7 Where Unix Began (https://bsdimp.blogspot.com/2019/07/the-pdp-7-where-unix-began.html) In preparation for a talk on Seventh Edition Unix this fall, I stumbled upon a service list from DEC for all known PDP-7 machines. From that list, and other sources, I believe that PDP-7 serial number 34 was the original Unix machine. V0 Unix could run on only one of the PDP-7s. Of the 99 PDP-7s produced, only two had disks. Serial number 14 had an RA01 listed, presumably a disk, though of a different type. In addition to the PDP-7 being obsolete in 1970, no other PDP-7 could run Unix, limiting its appeal outside of Bell Labs. By porting Unix to the PDP-11 in 1970, the group ensured Unix would live on into the future. The PDP-9 and PDP-15 were both upgrades of the PDP-7, so to be fair, PDP-7 Unix did have a natural upgrade path (the PDP-11 out sold the 18 bit systems though ~600,000 to ~1000). Ken Thompson reports in a private email that there were 2 PDP-9s and 1 PDP-15 at Bell Labs that could run a version of the PDP-7 Unix, though those machines were viewed as born obsolete. LLDB: watchpoints, XSTATE in ptrace() and core dumps (https://blog.netbsd.org/tnf/entry/lldb_watchpoints_xstate_in_ptrace) Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support and lately extending NetBSD's ptrace interface to cover more register types and fix compat32 issues. You can read more about that in my May 2019 report. In June, I have finally finished the remaining ptrace() work for xstate and got it merged both on NetBSD and LLDB end (meaning it's going to make it into NetBSD 9). I have also worked on debug register support in LLDB, effectively fixing watchpoint support. Once again I had to fight some upstream regressions. Beastie Bits Project Trident 19.07 Available (https://project-trident.org/post/2019-07-12_19.07_available/) A list of names from "Cold Blood" -- Any familiar? (https://www.montanalinux.org/cold-blood-list-of-numbers-201907.html) fern: a curses-based mastodon client modeled off usenet news readers & pine, with an emphasis on getting to 'timeline zero' (https://github.com/enkiv2/fern) OpenBSD Community goes Platinum for 2019! (https://undeadly.org/cgi?action=article;sid=20190707065226) tcp keepalive and dports on DragonFly (https://www.dragonflydigest.com/2019/07/15/23199.html) Feedback/Questions Patrick - OpenZFS/ZoL Module from Ports (http://dpaste.com/1W2HJ04) Brad - Services not starting (http://dpaste.com/345VM9Y#wrap) Simon - Feedback (http://dpaste.com/1B4ZKC8#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 24 Jul 2019 20:00:00 -0700
307: Twitching with OpenBSD
FreeBSD 11.3 has been released, OpenBSD workstation, write your own fuzzer for the NetBSD kernel, Exploiting FreeBSD-SA-19:02.fd, streaming to twitch using OpenBSD, 3 different ways of dumping hex contents of a file, and more. Headlines FreeBSD 11.3-RELEASE Announcement (https://www.freebsd.org/releases/11.3R/announce.html) The FreeBSD Release Engineering Team is pleased to announce the availability of FreeBSD 11.3-RELEASE. This is the fourth release of the stable/11 branch. Some of the highlights: The clang, llvm, lld, lldb, and compiler-rt utilities as well as libc++ have been updated to upstream version 8.0.0. The ELF Tool Chain has been updated to version r3614. OpenSSL has been updated to version 1.0.2s. The ZFS filesystem has been updated to implement parallel mounting. The loader(8) has been updated to extend geli(8) support to all architectures. The pkg(8) utility has been updated to version 1.10.5. The KDE desktop environment has been updated to version 5.15.3. The GNOME desktop environment has been updated to version 3.28. The kernel will now log the jail(8) ID when logging a process exit. Several feature additions and updates to userland applications. Several network driver firmware updates. Warnings for features deprecated in future releases will now be printed on all FreeBSD versions. Warnings have been added for IPSec algorithms deprecated in RFC 8221. Deprecation warnings have been added for weaker algorithms when creating geli(8) providers. And more... OpenBSD Is Now My Workstation (https://sogubsys.com/openbsd-is-now-my-workstation-operating-system/) Why OpenBSD? Simply because it is the best tool for the job for me for my new-to-me Lenovo Thinkpad T420. Additionally, I do care about security and non-bloat in my personal operating systems (business needs can have different priorities, to be clear). I will try to detail what my reasons are for going with OpenBSD (instead of GNU/Linux, NetBSD, or FreeBSD of which I’m comfortable using without issue), challenges and frustrations I’ve encountered, and what my opinions are along the way. Disclaimer: in this post, I’m speaking about what is my opinion, and I’m not trying to convince you to use OpenBSD or anything else. I don’t truly care, but wanted to share in case it could be useful to you. I do hope you give OpenBSD a shot as your workstation, especially if it has been a while. A Bit About Me and OpenBSD I’m not new to OpenBSD, to be clear. I’ve been using it off and on for over 20 years. The biggest time in my life was the early 2000s (I was even the Python port maintainer for a bit), where I not only used it for my workstation, but also for production servers and network devices. I just haven’t used it as a workstation (outside of a virtual machine) in over 10 years, but have used it for servers. Workstation needs, especially for a primary workstation, are greatly different and the small things end up mattering most. News Roundup Write your own fuzzer for NetBSD kernel! [Part 1] (https://blog.netbsd.org/tnf/entry/write_your_own_fuzzer_for) How Fuzzing works? The dummy Fuzzer. The easy way to describe fuzzing is to compare it to the process of unit testing a program, but with different input. This input can be random, or it can be generated in some way that makes it unexpected form standard execution perspective. The simplest 'fuzzer' can be written in few lines of bash, by getting N bytes from /dev/rand, and putting them to the program as a parameter. Coverage and Fuzzing What can be done to make fuzzing more effective? If we think about fuzzing as a process, where we place data into the input of the program (which is a black box), and we can only interact via input, not much more can be done. However, programs usually process different inputs at different speeds, which can give us some insight into the program's behavior. During fuzzing, we are trying to crash the program, thus we need additional probes to observe the program's behaviour. Additional knowledge about program state can be exploited as a feedback loop for generating new input vectors. Knowledge about the program itself and the structure of input data can also be considered. As an example, if the input data is in the form of HTML, changing characters inside the body will probably cause less problems for the parser than experimenting with headers and HTML tags. For open source programs, we can read the source code to know what input takes which execution path. Nonetheless, this might be very time consuming, and it would be much more helpful if this can be automated. As it turns out, this process can be improved by tracing coverage of the execution vBSDcon - CFP - Call for Papers ends July 19th (https://vbsdcon.com/) You can submit your proposal at https://easychair.org/conferences/?conf=vbsdcon2019 The talks will have a very strong technical content bias. Proposals of a business development or marketing nature are not appropriate for this venue. If you are doing something interesting with a BSD operating system, please submit a proposal. Whether you are developing a very complex system using BSD as the foundation, or helping others and have a story to tell about how BSD played a role, we want to hear about your experience. People using BSD as a platform for research are also encouraged to submit a proposal. Possible topics include: How we manage a giant installation with respect to handling spam, snd/or sysadmin, and/or networking, Cool new stuff in BSD, Tell us about your project which runs on BSD. Both users and developers are encouraged to share their experiences. Exploiting FreeBSD-SA-19:02.fd (https://secfault-security.com/blog/FreeBSD-SA-1902.fd.html) In February 2019 the FreeBSD project issued an advisory about a possible vulnerability in the handling of file descriptors. UNIX-like systems such as FreeBSD allow to send file descriptors to other processes via UNIX-domain sockets. This can for example be used to pass file access privileges to the receiving process. Inside the kernel, file descriptors are used to indirectly reference a C struct which stores the relevant information about the file object. This could for instance include a reference to a vnode which describes the file for the file system, the file type, or the access privileges. What really happens if a UNIX-domain socket is used to send a file descriptor to another process is that for the receiving process, inside the kernel a reference to this struct is created. As the new file descriptor is a reference to the same file object, all information is inherited. For instance, this can allow to give another process write access to a file on the drive even if the process owner is normally not able to open the file writable. The advisory describes that FreeBSD 12.0 introduced a bug in this mechanism. As the file descriptor information is sent via a socket, the sender and the receiver have to allocate buffers for the procedure. If the receiving buffer is not large enough, the FreeBSD kernel attempts to close the received file descriptors to prevent a leak of these to the sender. However, while the responsible function closes the file descriptor, it fails to release the reference from the file descriptor to the file object. This could cause the reference counter to wrap. The advisory further states that the impact of this bug is possibly a local privilege escalation to gain root privileges or a jail escape. However, no proof-of-concept was provided by the advisory authors. In the next section, the bug itself is analyzed to make a statement about the bug class and a guess about a possible exploitation primitive. After that, the bug trigger is addressed. It follows a discussion of three imaginable exploitation strategies - including a discussion of why two of these approaches failed. In the section before last, the working exploit primitive is discussed. It introduces a (at least to the author’s knowledge) new exploitation technique for these kind of vulnerabilities in FreeBSD. The stabilization of the exploit is addressed, too. The last section wraps everything up in a conclusion and points out further steps and challenges. The privilege escalation is now a piece of cake thanks to a technique used by kingcope, who published a FreeBSD root exploit in 2005, which writes to the file /etc/libmap.conf. This configuration file can be used to hook the loading of dynamic libraries if a program is started. The exploit therefore creates a dynamic library, which copies /bin/sh to another file and sets the suid-bit for the copy. The hooked library is libutil, which is for instance called by su. Therefore, a call to su by the user will afterwards result in a suid copy of /bin/sh. Streaming to Twitch using OpenBSD (https://dataswamp.org/~solene/2019-07-06-twitch.html) Introduction If you ever wanted to make a twitch stream from your OpenBSD system, this is now possible, thanks to OpenBSD developer thfr@ who made a wrapper named fauxstream using ffmpeg with relevant parameters. The setup is quite easy, it only requires a few steps and searching on Twitch website two informations, hopefully, to ease the process, I found the links for you. You will need to make an account on twitch, get your api key (a long string of characters) which should stay secret because it allow anyone having it to stream on your account. These same techniques should work for Twitch, YouTube Live, Periscope, Facebook, etc, including the live streaming service ScaleEngine provides free to BSD user groups. There is also an open source application called ‘OBS’ or Open Broadcaster Studio. It is in FreeBSD ports and should work on all of the other BSDs as well. It has a GUI and supports compositing and green screening. We use it heavily at ScaleEngine and it is also used at JupiterBroadcasting in place of WireCast, a $1000-per-copy commercial application. Beastie Bits Portland BSD Pizza Night - 2019-07-25 19:00 - Rudy's Gourmet Pizza (http://calagator.org/events/1250475868) KnoxBUG - Michael W. Lucas : Twenty Years in Jail (http://knoxbug.org/2019-07-29) Ohio Linuxfest - CFP - Closes August 17th (https://ohiolinux.org/call-for-presentations/) My college (NYU Tandon) is moving their CS department and I saw this on a shelf being moved (https://old.reddit.com/r/freebsd/comments/cdx8fp/my_college_nyu_tandon_is_moving_their_cs/) 3 different ways of dumping hex contents of a file (https://moopost.blogspot.com/2019/07/3-different-ways-of-dumping-hex.html) Feedback/Questions Sebastian - ZFS setup toward ESXi (http://dpaste.com/0DRKFH6#wrap) Christopher - Questions (http://dpaste.com/2YNN1SH) Ser - Bhyve and Microsoft SQL (http://dpaste.com/1F5TMT0#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 18 Jul 2019 07:00:00 -0700
306: Comparing Hammers
Am5x86 based retro UNIX build log, setting up services in a FreeNAS Jail, first taste of DragonflyBSD, streaming Netflix on NetBSD, NetBSD on the last G4 Mac mini, Hammer vs Hammer2, and more. Headlines Polprog's Am5x86 based retro UNIX build log (https://polprog.net/blog/486/) I have recently acquired an Am5x86 computer, in a surprisingly good condition. This is an ongoing project, check this page often for updates! I began by connecting a front panel. The panel came from a different chassis and is slightly too wide, so I had to attach it with a couple of zip-ties. However, that makes it stick out from the PC front at an angle, allowing easy access when the computer sits at the floor - and thats where it is most of the time. It's not that bad, to be honest, and its way easier to access than it would be, if mounted vertically There is a mains switch on the front panel because the computer uses an older style power supply. Those power supplies instead of relying on a PSON signal, like modern ATX supplies, run a 4 wire cable to a mains switch. The cable carries live and neutral both ways, and the switch keys in or out the power. The system powers on as soon as the switch is enabled. Originally there was no graphics card in it. Since a PC will not boot with out a GPU, I had to find one. The mainboard only has PCI and ISA slots, and all the GPUs I had were AGP. Fortunately, I bought a PCI GPU hoping it would solve my issue... However the GPU turned out to be faulty. It took me some time to repair it. I had to repair a broken trace leading to one of the EEPROM pins, and replace a contact in the EEPROM's socket. Then I replaced all the electrolytic capacitors on it, and that fixed it for good. Having used up only one of the three PCI slots, I populated the remaining pair with two ethernet cards. I still have a bunch of ISA slots available, but I have nothing to install there. Yet. See the article for the rest of the writeup Setting up services in a FreeNAS Jail (https://www.ixsystems.com/blog/services-in-freenas-jail/) This piece demonstrates the setup of a server service in a FreeNAS jail and how to share files with a jail using Apache 2.4 as an example. Jails are powerful, self-contained FreeBSD environments with separate network settings, package management, and access to thousands of FreeBSD application packages. Popular packages such as Apache, NGINX, LigHTTPD, MySQL, and PHP can be found and installed with the pkg search and pkg install commands. This example shows creating a jail, installing an Apache web server, and setting up a simple web page. NOTE: Do not directly attach FreeNAS to an external network (WAN). Use port forwarding, proper firewalls and DDoS protections when using FreeNAS for external web sites. This example demonstrates expanding the functionality of FreeNAS in an isolated LAN environment. News Roundup First taste of DragonflyBSD (https://nanxiao.me/en/first-taste-of-dragonfly-bsd/) Last week, I needed to pick a BSD Operating System which supports NUMA to do some testing, so I decided to give Dragonfly BSD a shot. Dragonfly BSDonly can run on X86_64 architecture, which reminds me of Arch Linux, and after some tweaking, I feel Dragonfly BSD may be a “developer-friendly” Operating System, at least for me. I mainly use Dragonfly BSD as a server, so I don’t care whether GUI is fancy or not. But I have high requirements of developer tools, i.e., compiler and debugger. The default compiler of Dragonfly BSD is gcc 8.3, and I can also install clang 8.0.0 from package. This means I can test state-of-the-art features of compilers, and it is really important for me. gdb‘s version is 7.6.1, a little lag behind, but still OK. Furthermore, the upgradation of Dragonfly BSD is pretty simple and straightforward. I followed document to upgrade my Operating System to 5.6.0 this morning, just copied and pasted, no single error, booted successfully. Streaming Netflix on NetBSD (https://www.unitedbsd.com/d/68-streaming-netflix-on-netbsd) Here's a step-by-step guide that allows streaming Netflix media on NetBSD using a intel-haxm accelerated QEMU vm. Heads-up! Sound doesn't work, but everything else is fine. Please read the rest of this thread for a solution to this!! “Sudo Mastery 2nd Edition” cover art reveal (https://mwl.io/archives/4320) I’m about halfway through the new edition of Sudo Mastery. Assuming nothing terrible happens, should have a complete first draft in four to six weeks. Enough stuff has changed in sudo that I need to carefully double-check every single feature. (I’m also horrified by the painfully obsolete versions of sudo shipped in the latest versions of CentOS and Debian, but people running those operating systems are already accustomed to their creaky obsolescence.) But the reason for this blog post? I have Eddie Sharam’s glorious cover art. My Patronizers saw it last month, so now the rest of you get a turn. NetBSD on the last G4 Mac mini (https://tenfourfox.blogspot.com/2019/06/and-now-for-something-completely.html) I'm a big fan of NetBSD. I've run it since 2000 on a Mac IIci (of course it's still running it) and I ran it for several years on a Power Mac 7300 with a G3 card which was the second incarnation of the Floodgap gopher server. Today I also still run it on a MIPS-based Cobalt RaQ 2 and an HP Jornada 690. I think NetBSD is a better match for smaller or underpowered systems than current-day Linux, and is fairly easy to harden and keep secure even though none of these systems are exposed to the outside world. Recently I had a need to set up a bridge system that would be fast enough to connect two networks and I happened to have two of the "secret" last-of-the-line 1.5GHz G4 Mac minis sitting on the shelf doing nothing. Yes, they're probably outclassed by later Raspberry Pi models, but I don't have to buy anything and I like putting old hardware to good use. Hammer vs Hammer2 (https://phoronix.com/scan.php?page=news_item&px=DragonFlyBSD-5.6-HAMMER2-Perf) With the newly released DragonFlyBSD 5.6 there are improvements to its original HAMMER2 file-system to the extent that it's now selected by its installer as the default file-system choice for new installations. Curious how the performance now compares between HAMMER and HAMMER2, here are some initial benchmarks on an NVMe solid-state drive using DragonFlyBSD 5.6.0. With a 120GB Toshiba NVMe SSD on an Intel Core i7 8700K system, I ran some benchmarks of DragonFlyBSD 5.6.0 freshly installed with HAMMER2 and then again when returning to the original HAMMER file-system that remains available via its installer. No other changes were made to the setup during testing. And then for the more synthetic workloads it was just a mix. But overall HAMMER2 was performing well during the initial testing and great to see it continuing to offer noticeable leads in real-world workloads compared to the aging HAMMER file-system. HAMMER2 also offers better clustering, online deduplication, snapshots, compression, encryption, and many other modern file-system features. Beastie Bits Unix CLI relational database (https://spin.atomicobject.com/2019/06/16/unix-cli-relational-database/) The TTY demystified (https://www.linusakesson.net/programming/tty/index.php) Ranger, a console file manager with VI keybindings (https://ranger.github.io/) Some Unix Humor (https://www.reddit.com/r/unix/comments/c6o5ze/some_unix_humor/) OpenBSD -import vulkan-loader for Vulkan API support (https://marc.info/?l=openbsd-ports-cvs&m=156121732625604&w=2) FreeBSD ZFS without drives (https://savagedlight.me/2019/06/09/freebsd-zfs-without-drives/) Feedback/Questions Moritz - ARM Builds (http://dpaste.com/175RRAZ) Dave - Videos (http://dpaste.com/2DYK85B) Chris - Raspberry Pi4 (http://dpaste.com/1B16QVN) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Thu, 11 Jul 2019 07:00:00 -0700
305: Changing face of Unix
Website protection with OPNsense, FreeBSD Support Pull Request for ZFS-on-Linux, How much has Unix changed, Porting Wine to amd64 on NetBSD, FreeBSD Enterprise 1 PB Storage, the death watch for X11 has started, and more. Headlines Website protection with OPNsense (https://medium.com/@jccwbb/website-protection-with-opnsense-3586a529d487) with nginx plugin OPNsense become a strong full featured Web Application Firewall (WAF) The OPNsense security platform can help you to protect your network and your webservers with the nginx plugin addition. In old days, install an open source firewall was a very trick task, but today it can be done with few clicks (or key strokes). In this article I'll not describe the detailed OPNsense installation process, but you can watch this video that was extracted from my OPNsense course available in Udemy. The video is in portuguese language, but with the translation CC Youtube feature you may be able to follow it without problems (if you don't are a portuguese speaker ofcourse) :-) + See the article for the rest of the writeup FreeBSD Support Pull Request against the ZFS-on-Linux repo (https://github.com/zfsonlinux/zfs/pull/8987) This pull request integrates the sysutils/openzfs port’s sources into the upstream ZoL repo > Adding FreeBSD support to ZoL will make it easier to move changes back and forth between FreeBSD and Linux > Refactor tree to separate out Linux and FreeBSD specific code > import FreeBSD's SPL > add ifdefs in common code where it made more sense to do so than duplicate the code in separate files > Adapted ZFS Test Suite to run on FreeBSD and all tests that pass on ZoL passing on ZoF The plan to officially rename the common repo from ZFSonLinux to OpenZFS was announced at the ZFS Leadership Meeting on June 25th Video of Leadership Meeting (https://www.youtube.com/watch?v=TJwykiJmH0M) Meeting Agenda and Notes (https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit) This will allow improvements made on one OS to be made available more easily (and more quickly) to the other platforms For example, mav@’s recent work: Add wakeupany(), cheaper version of wakeupone() for taskqueue(9) (https://svnweb.freebsd.org/base?view=revision&revision=349220) > As result, on 72-core Xeon v4 machine sequential ZFS write to 12 ZVOLs with 16KB block size spend 34% less time in wakeupany() and descendants then it was spending in wakeupone(), and total write throughput increased by ~10% with the same as before CPU usage. News Roundup Episode 5 Notes - How much has UNIX changed? (http://adventofcomputing.libsyn.com/episode-5-notes-how-much-has-unix-changed) UNIX-like systems have dominated computing for decades, and with the rise of the internet and mobile devices their reach has become even larger. True, most systems now use more modern OSs like Linux, but how much has the UNIX-like landscape changed since the early days? So, my question was this: how close is a modern *NIX userland to some of the earliest UNIX releases? To do this I'm going to compare a few key points of a modern Linux system with the earliest UNIX documentation I can get my hands on. The doc I am going to be covering(https://www.tuhs.org/Archive/Distributions/Research/Dennisv1/UNIXProgrammersManual_Nov71.pdf) is from November 1971, predating v1 of the system. I think the best place to start this comparison is to look at one of the highest-profile parts of the OS, that being the file system. Under the hood modern EXT file systems are completely different from the early UNIX file systems. However, they are still presented in basically the same way, as a heirerarchicat structure of directories with device files. So paths still look identical, and navigating the file system still functions the same. Often used commands like ls, cp, mv, du, and df function the same. So are mount and umount. But, there are some key differences. For instance, cd didn't exist, yet instead chdir filled its place. Also, chmod is somewhat different. Instead of the usual 3-digit octal codes for permissions, this older version only uses 2 digits. Really, that difference is due to the underlying file system using a different permission set than modern systems. For the most part, all the file handling is actually pretty close to a Linux system from 2019. See the article for the rest of the writeup Porting Wine to amd64 on NetBSD (https://blog.netbsd.org/tnf/entry/porting_wine_to_amd64_on) I have been working on porting Wine to amd64 on NetBSD as a GSoC 2019 project. Wine is a compatibility layer which allows running Microsoft Windows applications on POSIX-complaint operating systems. This report provides an overview of the progress of the project during the first coding period. Initially, when I started working on getting Wine-4.4 to build and run on NetBSD i386 the primary issue that I faced was Wine displaying black windows instead of UI, and this applied to any graphical program I tried running with Wine. I suspected it , as it is related to graphics, to be an issue with the graphics driver or Xorg. Subsequently, I tried building modular Xorg, and I tried running Wine on it only to realize that Xorg being modular didn't affect it in the least. After having tried a couple of configurations, I realized that trying to hazard out every other probability is going to take an awful lot of time that I didn't have. This motivated me to bisect the repo using git, and find the first version of Wine which failed on NetBSD. + See the article for the rest of the writeup FreeBSD Enterprise 1 PB Storage (https://vermaden.wordpress.com/2019/06/19/freebsd-enterprise-1-pb-storage/?utm_source=discoverbsd) Today FreeBSD operating system turns 26 years old. 19 June is an International FreeBSD Day. This is why I got something special today :). How about using FreeBSD as an Enterprise Storage solution on real hardware? This where FreeBSD shines with all its storage features ZFS included. Today I will show you how I have built so called Enterprise Storage based on FreeBSD system along with more then 1 PB (Petabyte) of raw capacity. This project is different. How much storage space can you squeeze from a single 4U system? It turns out a lot! Definitely more then 1 PB (1024 TB) of raw storage space. See the article for the rest of the writeup The death watch for the X Window System (aka X11) has probably started (https://utcc.utoronto.ca/~cks/space/blog/unix/XDeathwatchStarts) Once we are done with this we expect X.org to go into hard maintenance mode fairly quickly. The reality is that X.org is basically maintained by us and thus once we stop paying attention to it there is unlikely to be any major new releases coming out and there might even be some bitrot setting in over time. We will keep an eye on it as we will want to ensure X.org stays supportable until the end of the RHEL8 lifecycle at a minimum, but let this be a friendly notice for everyone who rely the work we do maintaining the Linux graphics stack, get onto Wayland, that is where the future is. I have no idea how true this is about X.org X server maintenance, either now or in the future, but I definitely think it's a sign that developers have started saying this. If Gnome developers feel that X.org is going to be in hard maintenance mode almost immediately, they're probably pretty likely to also put the Gnome code that deals with X into hard maintenance mode. And public Gnome statements about this (and public action or lack of it) provide implicit support for KDE and any other desktop to move in this direction if they want to (and probably create some pressure to do so). I've known that Wayland was the future for some time, but I would still like it to not arrive any time soon. Beastie Bits Porting NetBSD to Risc-V -- Video (https://www.youtube.com/watch?v=2vQXGomKoxA) FreeBSD 11.3RC3 Available (https://www.freebsd.org/news/newsflash.html#event20190628:01) Open Source Could Be a Casualty of the Trade War (https://www.bunniestudios.com/blog/?p=5590) Celebrate UNIX50 and SDF32 (https://sdf.org/sdf32/) doas environmental security (https://undeadly.org/cgi?action=article;sid=20190621104048) Feedback/Questions Matt - BSD or Older Hardware (http://dpaste.com/1RP09F0#wrap) MJRodriguez - Some Playstation news (http://dpaste.com/046SPPB#wrap) Moritz - bhyve VT-x passthrough (http://dpaste.com/1H4PJXW) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 03 Jul 2019 19:00:00 -0700
304: Prospering with Vulkan
DragonflyBSD 5.6 is out, OpenBSD Vulkan Support, bad utmp implementations in glibc and FreeBSD, OpenSSH protects itself against Side Channel attacks, ZFS vs OpenZFS, and more. Headlines DragonflyBSD 5.6 is out (https://www.dragonflybsd.org/release56) Version 5.6.0 released 17 June 2019 Version 5.6.1 released 19 June 2019 (https://www.dragonflydigest.com/2019/06/19/23091.html) Big-ticket items Improved VM Informal test results showing the changes from 5.4 to 5.6 are available. Reduce stalls in the kernel vmpagealloc() code (vmpagelist_find()). Improve page allocation algorithm to avoid re-iterating the same queues as the search is widened. Add a vmpagehash*() API that allows the kernel to do heuristical lockless lookups of VM pages. Change vmhold() and vmunhold() semantics to not require any spin-locks. Change vmpagewakeup() to not require any spin-locks. Change wiring vm_page's no longer manipulates the queue the page is on, saving a lot of overhead. Instead, the page will be removed from its queue only if the pageout demon encounters it. This allows pages to enter and leave the buffer cache quickly. Refactor the handling of fictitious pages. Remove m->md.pvlist entirely. VM pages in mappings no longer allocate pventry's, saving an enormous amount of memory when multiple processes utilize large shared memory maps (e.g. postgres database cache). Refactor vmobject shadowing, disconnecting the backing linkages from the vmobject itself and instead organizing the linkages in a new structure called vmmapbacking which hangs off the vmmapentry. pmap operations now iterate vmmapbacking structures (rather than spin-locked page lists based on the vmpage and pventry's), and will test/match operations against the PTE found in the pmap at the requisite location. This doubles VM fault performance on shared pages and reduces the locking overhead for fault and pmap operations. Simplify the collapse code, removing most of the original code and replacing it with simpler per-vmmapentry optimizations to limit the shadow depth. DRM Major updates to the radeon and ttm (amd support code) drivers. We have not quite gotten the AMD support up to the more modern cards or Ryzen APUs yet, however. Improve UEFI framebuffer support. A major deadlock has been fixed in the radeon/ttm code. Refactor the startup delay designed to avoid conflicts between the i915 driver initialization and X startup. Add DRMIOCTLGET_PCIINFO to improve mesa/libdrm support. Fix excessive wired memory build-ups. Fix Linux/DragonFly PAGE_MASK confusion in the DRM code. Fix idr_*() API bugs. HAMMER2 The filesystem sync code has been rewritten to significantly improve performance. Sequential write performance also improved. Add simple dependency tracking to prevent directory/file splits during create/rename/remove operations, for better consistency after a crash. Refactor the snapshot code to reduce flush latency and to ensure a consistent snapshot. Attempt to pipeline the flush code against the frontend, improving flush vs frontend write concurrency. Improve umount operation. Fix an allocator race that could lead to corruption. Numerous other bugs fixed. Improve verbosity of CHECK (CRC error) console messages. OpenBSD Vulkan Support (https://www.phoronix.com/scan.php?page=news_item&px=OpenBSD-Vulkan-Support) Somewhat surprisingly, OpenBSD has added the Vulkan library and ICD loader support as their newest port. This new graphics/vulkan-loader port provides the generic Vulkan library and ICD support that is the common code for Vulkan implementations on the system. This doesn't enable any Vulkan hardware drivers or provide something new not available elsewhere, but is rare seeing Vulkan work among the BSDs. There is also in ports the related components like the SPIR-V headers and tools, glsllang, and the Vulkan tools and validation layers. This is of limited usefulness, at least for the time being considering OpenBSD like the other BSDs lag behind in their DRM kernel driver support that is ported over from the mainline Linux kernel tree but generally years behind the kernel upstream. Particularly with Vulkan, newer kernel releases are needed for some Vulkan features as well as achieving decent performance. The Vulkan drivers of relevance are the open-source Intel ANV Vulkan driver and Radeon RADV drivers, both of which are in Mesa though we haven't seen any testing results to know how well they would work if at all currently on OpenBSD, but they're at least in Mesa and obviously open-source. + A note: The BSDs are no longer that far behind. + FreeBSD 12.0 uses DRM from Linux 4.16 (April 2018), and the drm-devel port is based on Linux 5.0 (March 2019) + OpenBSD -current as of April 2019 uses DRM from Linux 4.19.34 News Roundup Bad utmp implementations in glibc and freebsd (https://davmac.wordpress.com/2019/05/04/bad-utmp-implementations-in-glibc-and-freebsd/) I recently released another version – 0.5.0 – of Dinit, the service manager / init system. There were a number of minor improvements, including to the build system (just running “make” or “gmake” should be enough on any of the systems which have a pre-defined configuration, no need to edit mconfig by hand), but the main features of the release were S6-compatible readiness notification, and support for updating the utmp database. In other words, utmp is a record of who is currently logged in to the system (another file, “wtmp”, records all logins and logouts, as well as, potentially, certain system events such as reboots and time updates). This is a hint at the main motivation for having utmp support in Dinit – I wanted the “who” command to correctly report current logins (and I wanted boot time to be correctly recorded in the wtmp file). I wondered: If the files consist of fixed-sized records, and are readable by regular users, how is consistency maintained? That is – how can a process ensure that, when it updates the database, it doesn’t conflict with another process also attempting to update the database at the same time? Similarly, how can a process reading an entry from the database be sure that it receives a consistent, full record and not a record which has been partially updated? (after all, POSIX allows that a write(2) call can return without having written all the requested bytes, and I’m not aware of Linux or any of the *BSDs documenting that this cannot happen for regular files). Clearly, some kind of locking is needed; a process that wants to write to or read from the database locks it first, performs its operation, and then unlocks the database. Once again, this happens under the hood, in the implementation of the getutent/pututline functions or their equivalents. Then I wondered: if a user process is able to lock the utmp file, and this prevents updates, what’s to stop a user process from manually acquiring and then holding such a lock for a long – even practically infinite – duration? This would prevent the database from being updated, and would perhaps even prevent logins/logouts from completing. Unfortunately, the answer is – nothing; and yes, it is possible on different systems to prevent the database from being correctly updated or even to prevent all other users – including root – from logging in to the system. + A good find + On FreeBSD, even though write(2) can be asynchronous, once the write syscall returns, the data is in the buffer cache (or ARC), and any future read(2) will see that new data even if it has not yet been written to disk. OpenSSH gets an update to protect against Side Channel attacks (https://securityboulevard.com/2019/06/openssh-code-gets-an-update-to-protect-against-side-channel-attacks/) Last week, Damien Miller, a Google security researcher, and one of the popular OpenSSH and OpenBSD developers announced an update to the existing OpenSSH code that can help protect against the side-channel attacks that leak sensitive data from computer’s memory. This protection, Miller says, will protect the private keys residing in the RAM against Spectre, Meltdown, Rowhammer, and the latest RAMBleed attack. SSH private keys can be used by malicious threat actors to connect to remote servers without the need of a password. According to CSO, “The approach used by OpenSSH could be copied by other software projects to protect their own keys and secrets in memory”. However, if the attacker is successful in extracting the data from a computer or server’s RAM, they will only obtain an encrypted version of an SSH private key, rather than the cleartext version. In an email to OpenBSD, Miller writes, “this change encrypts private keys when they are not in use with a symmetric key that is derived from a relatively large ‘prekey’ consisting of random data (currently 16KB).” ZFS vs OpenZFS (https://www.ixsystems.com/blog/zfs-vs-openzfs/) You’ve probably heard us say a mix of “ZFS” and “OpenZFS” and an explanation is long-overdue. From its inception, “ZFS” has referred to the “Zettabyte File System” developed at Sun Microsystems and published under the CDDL Open Source license in 2005 as part of the OpenSolaris operating system. ZFS was revolutionary for completely decoupling the file system from specialized storage hardware and even a specific computer platform. The portable nature and advanced features of ZFS led FreeBSD, Linux, and even Apple developers to start porting ZFS to their operating systems and by 2008, FreeBSD shipped with ZFS in the 7.0 release. For the first time, ZFS empowered users of any budget with enterprise-class scalability and data integrity and management features like checksumming, compression and snapshotting, and those features remain unrivaled at any price to this day. On any ZFS platform, administrators use the zpool and zfs utilities to configure and manage their storage devices and file systems respectively. Both commands employ a user-friendly syntax such as‘zfs create mypool/mydataset’ and I welcome you to watch the appropriately-titled webinar “Why we love ZFS & you should too” or try a completely-graphical ZFS experience with FreeNAS. Oracle has steadily continued to develop its own proprietary branch of ZFS and Matt Ahrens points out that over 50% of the original OpenSolaris ZFS code has been replaced in OpenZFS with community contributions. This means that there are, sadly, two politically and technologically-incompatible branches of “ZFS” but fortunately, OpenZFS is orders of magnitude more popular thanks to its open nature. The two projects should be referred to as “Oracle ZFS” and “OpenZFS” to distinguish them as development efforts, but the user still types the ‘zfs’ command, which on FreeBSD relies on the ‘zfs.ko’ kernel module. My impression is that the terms of the CDDL license under which the OpenZFS branch of ZFS is published protects its users from any patent and trademark risks. Hopefully, this all helps you distinguish the OpenZFS project from the ZFS technology. + There was further discussion of how the ZFSOnLinux repo will become the OpenZFS repo in the future once it also contains the bits to build on FreeBSD as well during the June 25th ZFS Leadership Meeting. The videos for all of the meetings are available here (https://www.youtube.com/channel/UC0IK6Y4Go2KtRueHDiQcxow) Beastie Bits How to safely and portably close a file descriptor in a multithreaded process without running into problems with EINTR (https://twitter.com/cperciva/status/1141852451756105729?s=03) KnoxBug Meetup June 27th at 6pm (http://knoxbug.org/2019-06-27) BSD Pizza Night, June 27th at 7pm, Flying Pie Pizzeria, 3 Monroe Pkwy, Ste S, Lake Oswego, OR (https://www.flying-pie.com/locations/lake-oswego/) Difference between $x and ${x} (https://moopost.blogspot.com/2019/06/difference-between-x-and-x.html) Beware of Software Engineering Media Sites (https://www.nemil.com/on-software-engineering/beware-engineering-media.html) How Verizon and a BGP optimizer knocked large parts of the internet offline today (https://blog.cloudflare.com/how-verizon-and-a-bgp-optimizer-knocked-large-parts-of-the-internet-offline-today/) DragonflyBSD - MDS mitigation added a while ago (http://lists.dragonflybsd.org/pipermail/commits/2019-May/718899.html) Reminder: Register for EuroBSDcon 2019 in Lillehammer, Norway (https://eurobsdcon.org) Feedback/Questions Dave - CheriBSD (http://dpaste.com/38233JC) Neb - Hello from Norway (http://dpaste.com/0B8XKXT#wrap) Lars - Ansible tutorial? (http://dpaste.com/3N85SHR) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** Your browser does not support the HTML5 video tag.
Thu, 27 Jun 2019 00:45:00 -0700
303: OpenZFS in Ports
OpenZFS-kmod port available, using blacklistd with NPF as fail2ban replacement, ZFS raidz expansion alpha preview 1, audio VU-meter increases CO2 footprint rant, XSAVE and compat32 kernel work for LLDB, where icons for modern X applications come from, and more. Headlines ZFSonFreeBSD ports renamed OpenZFS (https://www.freshports.org/sysutils/openzfs-kmod) The ZFS on FreeBSD project has renamed the userland and kernel ports from zol and zol-kmod to openzfs and openzfs-kmod The new versions from this week are IOCTL compatible with the command line tools in FreeBSD 12.0, so you can use the old userland with the new kernel module (although obviously not the new features) With the renaming it is easier to specify which kernel module you want to load in /boot/loader.conf: > zfs_load=”YES” or > openzfs_load=”YES” To load traditional or the newer version of ZFS The kmod still requires FreeBSD 12-stable or 13-current because it depends on the newer crypto support in the kernel for the ZFS native encryption feature. Allan is looking at ways to work around this, but it may not be practical. We would like to do an unofficial poll on how people would the userland to co-exist. Add a suffix to the new commands in /usr/local (zfs.new zpool.new or whatever). One idea i’ve had is to move the zfs and zpool commands to /libexec and make /sbin/zfs and /sbin/zpool a switcher script, that will call the base or ports version based on a config file (or just based on if the port is installed) For testing purposes, generally you should be fine as long as you don’t run ‘zpool upgrade’, which will make your pool only importable using the newer ZFS. For extra safety, you can create a ‘zpool checkpoint’, which will allow you to undo any changes that are made to the pool during your testing with the new openzfs tools. Note: the checkpoint will undo EVERYTHING. So don’t save new data you want to keep. Note: Checkpoints disable all freeing operations, to prevent any data from being overwritten so that you can re-import at the checkpoint and undo any operation (including zfs destroy-ing a dataset), so also be careful you don’t run out of space during testing. Please test and provide feedback. How to use blacklistd(8) with NPF as a fail2ban replacement (https://www.unitedbsd.com/d/63-how-to-use-blacklistd8-with-npf-as-a-fail2ban-replacement) About blacklistd(8) blacklistd(8) provides an API that can be used by network daemons to communicate with a packet filter via a daemon to enforce opening and closing ports dynamically based on policy. The interface to the packet filter is in /libexec/blacklistd-helper (this is currently designed for npf) and the configuration file (inspired from inetd.conf) is in etc/blacklistd.conf Now, blacklistd(8) will require bpfjit(4) (Just-In-Time compiler for Berkeley Packet Filter) in order to properly work, in addition to, naturally, npf(7) as frontend and syslogd(8), as a backend to print diagnostic messages. Also remember npf shall rely on the npflog* virtual network interface to provide logging for tcpdump() to use. Unfortunately (dont' ask me why :P) in 8.1 all the required kernel components are still not compiled by default in the GENERIC kernel (though they are in HEAD), and are rather provided as modules. Enabling NPF and blacklistd services would normally result in them being automatically loaded as root, but predictably on securelevel=1 this is not going to happen News Roundup [WIP] raidz expansion, alpha preview 1 (https://github.com/zfsonlinux/zfs/pull/8853) Motivation and Context > This is a alpha-quality preview of RAID-Z expansion. This feature allows disks to be added one at a time to a RAID-Z group, expanding its capacity incrementally. This feature is especially useful for small pools (typically with only one RAID-Z group), where there isn't sufficient hardware to add capacity by adding a whole new RAID-Z group (typically doubling the number of disks). > For additional context as well as a design overview, see my short talk from the 2017 OpenZFS Developer Summit: slides video Rant: running audio VU-meter increases my CO2 footprint (https://medium.com/@MartinCracauer/bug-rant-running-audio-vu-meter-increases-my-co2-footprint-871d5c1bee5a) A couple months ago I noticed that the monitor on my workstation never power off anymore. Screensaver would go on, but DPMs (to do the poweroff) never kicked in. I grovels the output of various tools that display DPMS settings, which as usual in Xorg were useless. Everybody said DPMS is on with a timeout. I even wrote my own C program to use every available Xlib API call and even the xscreensaver library calls. (should make it available) No go, everybody says that DPMs is on, enabled and set on a timeout. Didn’t matter whether I let xscreeensaver do the job or just the X11 server. After a while I noticed that DPMS actually worked between starting my X11 server and starting all my clients. I have a minimal .xinitrc and start the actual session from a script, that is how I could notice. If I used a regular desktop login I wouldn’t have noticed. A server state bug was much more likely than a client bug. + See the article for the rest... XSAVE and compat32 kernel work for LLDB (http://blog.netbsd.org/tnf/entry/xsave_and_compat32_kernel_work) Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support and lately extending NetBSD's ptrace interface to cover more register types. You can read more about that in my Apr 2019 report. In May, I was primarily continuing the work on new ptrace interface. Besides that, I've found and fixed a bug in ptrace() compat32 code, pushed LLVM buildbot to ‘green’ status and found some upstream LLVM regressions. More below. Some things about where icons for modern X applications come from (https://utcc.utoronto.ca/~cks/space/blog/unix/ModernXAppIcons) If you have a traditional window manager like fvwm, one of the things it can do is iconify X windows so that they turn into icons on the root window (which would often be called the 'desktop'). Even modern desktop environments that don't iconify programs to the root window (or their desktop) may have per-program icons for running programs in their dock or taskbar. If your window manager or desktop environment can do this, you might reasonably wonder where those icons come from by default. Although I don't know how it was done in the early days of X, the modern standard for this is part of the Extended Window Manager Hints. In EWMH, applications give the window manager a number of possible icons, generally in different sizes, as ARGB bitmaps (instead of, say, SVG format). The window manager or desktop environment can then pick whichever icon size it likes best, taking into account things like the display resolution and so on, and display it however it wants to (in its original size or scaled up or down). How this is communicated in specific is through the only good interprocess communication method that X supplies, namely X properties. In the specific case of icons, the NETWMICON property is what is used, and xprop can display the size information and an ASCII art summary of what each icon looks like. It's also possible to use some additional magic to read out the raw data from _NETWM_ICON in a useful format; see, for example, this Stackoverflow question and its answers. Beastie Bits Recent Security Innovations (http://undeadly.org/cgi?action=article;sid=20190605110020) Old Unix books + Solaris (https://imgur.com/a/HbSYtQI) Pro-Desktop - A Tiling Desktop Environment (https://bitcannon.net/post/pro-desktop/) The Tar Pipe (https://blog.extracheese.org/2010/05/the-tar-pipe.html) At least one vim trick you might not know (https://www.hillelwayne.com/post/intermediate-vim/) Feedback/Questions Johnny - listener feedback (http://dpaste.com/0ZQCQ8Y#wrap) Brian - Questions (http://dpaste.com/1843RNX#wrap) Mark - ZFS Question (http://dpaste.com/3M83X9G#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
Wed, 19 Jun 2019 19:30:00 -0700
302: Contention Reduction
DragonFlyBSD's kernel optimizations pay off, differences between OpenBSD and Linux, NetBSD 2019 Google Summer of Code project list, Reducing that contention, fnaify 1.3 released, vmctl(8): CLI syntax changes, and things that Linux distributions should not do when packaging. Headlines DragonFlyBSD's Kernel Optimizations Are Paying Off (https://www.phoronix.com/scan.php?page=article&item=dragonfly-55-threadripper&num=1) DragonFlyBSD lead developer Matthew Dillon has been working on a big VM rework in the name of performance and other kernel improvements recently. Here is a look at how those DragonFlyBSD 5.5-DEVELOPMENT improvements are paying off compared to DragonFlyBSD 5.4 as well as FreeBSD 12 and five Linux distribution releases. With Dillon using an AMD Ryzen Threadripper system, we used that too for this round of BSD vs. Linux performance benchmarks. The work by Dillon on the VM overhaul and other changes (including more HAMMER2 file-system work) will ultimately culminate with the DragonFlyBSD 5.6 release (well, unless he opts for DragonFlyBSD 6.0 or so). These are benchmarks of the latest DragonFlyBSD 5.5-DEVELOPMENT daily ISO as of this week benchmarked across DragonFlyBSD 5.4.3 stable, FreeBSD 12.0, Ubuntu 19.04, Red Hat Enterprise Linux 8.0, Debian 9.9, Debian Buster, and CentOS 7 1810 as a wide variety of reference points both from newer and older Linux distributions. (As for no Clear Linux reference point for a speedy reference point, it currently has a regression with AMD + Samsung NVMe SSD support on some hardware, including this box, prohibiting the drive from coming up due to a presumed power management issue that is still being resolved.) With Matthew Dillon doing much of his development on an AMD Ryzen Threadripper system after he last year proclaimed the greatness of these AMD HEDT CPUs, for this round of testing I also used a Ryzen Threadripper 2990WX with 32 cores / 64 threads. Tests of other AMD/Intel hardware with DragonFlyBSD will come as the next stable release is near and all of the kernel work has settled down. For now it's mostly entertaining our own curiosity how well these DragonFlyBSD optimizations are paying off and how it's increasing the competition against FreeBSD 12 and Linux distributions. What are the differences between OpenBSD and Linux? (https://cfenollosa.com/blog/what-are-the-differences-between-openbsd-and-linux.html) Maybe you have been reading recently about the release of OpenBSD 6.5 and wonder, "What are the differences between Linux and OpenBSD?" I've also been there at some point in the past and these are my conclusions. They also apply, to some extent, to other BSDs. However, an important disclaimer applies to this article. This list is aimed at people who are used to Linux and are curious about OpenBSD. It is written to highlight the most important changes from their perspective, not the absolute most important changes from a technical standpoint. Please bear with me. A terminal is a terminal is a terminal Practical differences Security and system administration Why philosophical differences matter So what do I choose? How to try OpenBSD *** News Roundup NetBSD 2019 Google Summer of Code (http://blog.netbsd.org/tnf/entry/announcing_google_summer_of_code1) We are very happy to announce The NetBSD Foundation Google Summer of Code 2019 projects: Akul Abhilash Pillai - Adapting TriforceAFL for NetBSD kernel fuzzing Manikishan Ghantasala - Add KNF (NetBSD style) clang-format configuration Siddharth Muralee - Enhancing Syzkaller support for NetBSD Surya P - Implementation of COMPATLINUX and COMPATNETBSD32 DRM ioctls support for NetBSD kernel Jason High - Incorporation of Argon2 Password Hashing Algorithm into NetBSD Saurav Prakash - Porting NetBSD to HummingBoard Pulse Naveen Narayanan - Porting WINE to amd64 architecture on NetBSD The communiting bonding period - where students get in touch with mentors and community - started yesterday. The coding period will start from May 27 until August 19. Please welcome all our students and a big good luck to students and mentors! A big thank to Google and The NetBSD Foundation organization mentors and administrators! Looking forward to a great Google Summer of Code! Reducing that contention (http://www.grenadille.net/post/2019/05/09/Reducing-that-contention) The opening keynote at EuroBSDCon 2016 predicted the future 10 years of BSDs. Amongst all the funny previsions, gnn@FreeBSD said that by 2026 OpenBSD will have its first implementation of SMP. Almost 3 years after this talk, that sounds like a plausible forecast... Why? Where are we? What can we do? Let's dive into the issue! State of affairs Most of OpenBSD's kernel still runs under a single lock, ze KERNEL_LOCK(). That includes most of the syscalls, most of the interrupt handlers and most of the fault handlers. Most of them, not all of them. Meaning we have collected & fixed bugs while setting up infrastructures and examples. Now this lock remains the principal responsible for the spin % you can observe in top(1) and systat(1). I believe that we opted for a difficult hike when we decided to start removing this lock from the bottom. As a result many SCSI & Network interrupt handlers as well as all Audio & USB ones can be executed without big lock. On the other hand very few syscalls are already or almost ready to be unlocked, as we incorrectly say. This explains why basic primitives like tsleep(9), csignal() and selwakeup() are only receiving attention now that the top of the Network Stack is running (mostly) without big lock. Next steps In the past years, most of our efforts have been invested into the Network Stack. As I already mentioned it should be ready to be parallelized. However think we should now concentrate on removing the KERNEL_LOCK(), even if the code paths aren't performance critical. See the Article for the rest of the post fnaify 1.3 released - more games are "fnaify & run" now (https://www.reddit.com/r/openbsd_gaming/comments/btste9/fnaify_13_released_more_games_are_fnaify_run_now/) This release finally addresses some of the problems that prevent simple running of several games. This happens for example when an old FNA.dll library comes with the games that doesn't match the API of our native libraries like SDL2, OpenAL, or MojoShader anymore. Some of those cases can be fixed by simply dropping in a newer FNA.dll. fnaify now asks if FNA 17.12 should be automatically added if a known incompatible FNA version is found. You simply answer yes or no. Another blocker happens when the game expects to check the SteamAPI - either from a running Steam process, or a bundled steam_api library. OpenBSD 6.5-current now has steamworks-nosteam in ports, a stub library for Steamworks.NET that prevents games from crashing simply because an API function isn't found. The repo is here. fnaify now finds this library in /usr/local/share/steamstubs and uses it instead of the bundled (full) Steamworks.NET.dll. This may help with any games that use this layer to interact with the SteamAPI, mostly those that can only be obtained via Steam. vmctl(8): command line syntax changed (https://www.openbsd.org/faq/current.html#r20190529) The order of the arguments in the create, start, and stop commands of vmctl(8) has been changed to match a commonly expected style. Manual usage or scripting with vmctl must be adjusted to use the new syntax. For example, the old syntax looked like this: # vmctl create disk.qcow2 -s 50G The new syntax specifies the command options before the argument: # vmctl create -s 50G disk.qcow2 Something that Linux distributions should not do when packaging things (https://utcc.utoronto.ca/~cks/space/blog/linux/PackageNameClashProblem) Right now I am a bit unhappy at Fedora for a specific packaging situation, so let me tell you a little story of what I, as a system administrator, would really like distributions to not do. For reasons beyond the scope of this blog entry, I run a Prometheus and Grafana setup on both my home and office Fedora Linux machines (among other things, it gives me a place to test out various things involving them). When I set this up, I used the official upstream versions of both, because I needed to match what we are running (or would soon be). Recently, Fedora decided to package Grafana themselves (as a RPM), and they called this RPM package 'grafana'. Since the two different packages are different versions of the same thing as far as package management tools are concerned, Fedora basically took over the 'grafana' package name from Grafana. This caused my systems to offer to upgrade me from the Grafana.com 'grafana-6.1.5-1' package to the Fedora 'grafana-6.1.6-1.fc29' one, which I actually did after taking reasonable steps to make sure that the Fedora version of 6.1.6 was compatible with the file layouts and so on from the Grafana version of 6.1.5. Why is this a problem? It's simple. If you're going to take over a package name from the upstream, you should keep up with the upstream releases. If you take over a package name and don't keep up to date or keep up to date only sporadically, you cause all sorts of heartburn for system administrators who use the package. The least annoying future of this situation is that Fedora has abandoned Grafana at 6.1.6 and I am going to 'upgrade' it with the upstream 6.2.1, which will hopefully be a transparent replacement and not blow up in my face. The most annoying future is that Fedora and Grafana keep ping-ponging versions back and forth, which will make 'dnf upgrade' into a minefield (because it will frequently try to give me a 'grafana' upgrade that I don't want and that would be dangerous to accept). And of course this situation turns Fedora version upgrades into their own minefield, since now I risk an upgrade to Fedora 30 actually reverting the 'grafana' package version on me. Beastie Bits [talk] ZFS v UFS on APU2 msata SSD with FreeBSD (http://lists.nycbug.org:8080/pipermail/talk/2019-May/017885.html) NetBSD 8.1 is out (http://www.netbsd.org/releases/formal-8/NetBSD-8.1.html) lazyboi – the laziest possible way to send raw HTTP POST data (https://github.com/ctsrc/lazyboi) A Keyboard layout that changes by markov frequency (https://github.com/shapr/markovkeyboard) Open Source Game Clones (https://osgameclones.com/) EuroBSDcon program & registration open (https://eurobsdcon.org) *** Feedback/Questions John - A segment idea (http://dpaste.com/3YTBQTX#wrap) Johnny - Audio only format please don't (http://dpaste.com/3WD0A25#wrap) Alex - Thanks and some Linux Snaps vs PBI feedback (http://dpaste.com/1RQF4QM#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** Your browser does not support the HTML5 video tag.
Wed, 12 Jun 2019 20:00:00 -0700
301: GPU Passthrough
GPU passthrough on bhyve, confusion with used/free disk space on ZFS, OmniOS Community Edition, pfSense 2.4.4 Release p3, NetBSD 8.1 RC1, FreeNAS as your Server OS, and more. Headlines GPU Passthrough Reported Working on Bhyve

Normally we cover news focused on KVM and sometimes Xen, but something very special has happened with their younger cousin in the BSD world, Bhyve. For those that don’t know, Bhyve (pronounced bee-hive) is the native hypervisor in FreeBSD. It has many powerful features, but one that’s been a pain point for some years now is VGA passthrough. Consumer GPUs have not been useable until very recently despite limited success with enterprise cards. However, Twitter user Michael Yuji found a workaround that enables passing through a consumer card to any *nix system configured to use X11:

  • https://twitter.com/michael_yuji/status/1127136891365658625

All you have to do is add a line pointing the X server to the Bus ID of the passed card and the VM will boot, with acceleration and everything. He theorizes that this may not be possible on windows because of the way it looks for display devices, but it’s a solid start. As soon as development surrounding VGA passthrough matures on Bhyve, it will become a very attractive alternative to more common tools like Hyper-V and Qemu, because it makes many powerful features available in the host system like jails, boot environments, BSD networking, and tight ZFS integration. For example, you could potentially run your Router, NAS, preferred workstation OS and any number of other things in one box, and only have to spin up a single VM because of the flexibility afforded by jails over Linux-based containers. The user who found this workaround also announced they’d be writing it up at some point, so stay tuned for details on the process. It’s been slow going on Bhyve passthrough development for a while, but this new revelation is encouraging. We’ll be closely monitoring the situation and report on any other happenings.

Confusion with used/free disk space in ZFS

I use ZFS extensively. ZFS is my favorite file system. I write articles and give lectures about it. I work with it every day. In traditional file systems we use df(1) to determine free space on partitions. We can also use du(1) to count the size of the files in the directory. But it’s different on ZFS and this is the most confusing thing EVER. I always forget which tool reports what disk space usage! Every time somebody asks me, I need to google it. For this reason I decided to document it here - for myself - because if I can’t remember it at least I will not need to google it, as it will be on my blog, but maybe you will also benefit from this blog post if you have the same problem or you are starting your journey with ZFS.

The understanding of how ZFS is uses space and how to determine which value means what is a crucial thing. I hope thanks to this article I will finally remember it!

News Roundup OmniOS Community Edition

The OmniOS Community Edition Association is proud to announce the general availability of OmniOS - r151030. OmniOS is published according to a 6-month release cycle, r151030 LTS takes over from r151028, published in November 2018; and since it is a LTS release it also takes over from r151022. The r151030 LTS release will be supported for 3 Years. It is the first LTS release published by the OmniOS CE Association since taking over the reins from OmniTI in 2017. The next LTS release is scheduled for May 2021. The old stable r151026 release is now end-of-life. See the release schedule for further details. This is only a small selection of the new features, and bug fixes in the new release; review the release notes for full details. If you upgrade from r22 and want to see all new features added since then, make sure to also read the release notes for r24, r26 and r28. The OmniOS team and the illumos community have been very active in creating new features and improving existing ones over the last 6 months.

pfSense 2.4.4 Release p3 is available

We are pleased to announce the release of pfSense® software version 2.4.4-p3, now available for new installations and upgrades! pfSense software version 2.4.4-p3 is a maintenance release, bringing a number of security enhancements as well as a handful of fixes for issues present in the 2.4.4-p2 release. pfSense 2.4.4-RELEASE-p3 updates and installation images are available now! To see a complete list of changes and find more detail, see the Release Notes. We had hoped to bring you this release a few days earlier, but given the announcement last Tuesday of the Intel Microarchitectural Data Sampling (MDS) issue, we did not have sufficient time to fully incorporate those corrections and properly test for release on Thursday. We felt that it was worth delaying for a few days, rather than making multiple releases within a week.

  • Upgrade Notes

Due to the significant nature of the changes in 2.4.4 and later, warnings and error messages, particularly from PHP and package updates, are likely to occur during the upgrade process. In nearly all cases these errors are a harmless side effect of the changes between FreeBSD 11.1 and 11.2 and between PHP 5.6 and PHP 7.2. Always take a backup of the firewall configuration prior to any major change to the firewall, such as an upgrade. Do not update packages before upgrading pfSense! Either remove all packages or do not update packages before running the upgrade. The upgrade will take several minutes to complete. The exact time varies based on download speed, hardware speed, and other factors such installed packages. Be patient during the upgrade and allow the firewall enough time to complete the entire process. After the update packages finish downloading it could take 10-20 minutes or more until the upgrade process ends. The firewall may reboot several times during the upgrade process. Monitor the upgrade from the firewall console for the most accurate view.

NetBSD 8.1 RC1 is out

The NetBSD Project is pleased to announce NetBSD 8.1, the first update of the NetBSD 8 release branch. It represents a selected subset of fixes deemed important for security or stability reasons, as well as new features and enhancements.

Some highlights of the 8.1 release are:

  • x86: Mitigation for INTEL-SA-00233 (MDS)
  • Various local user kernel data leaks fixed.
  • x86: new rc.conf(5) setting smtoff to disable Simultaneous Multi-Threading
  • Various network driver fixes and improvements.
  • Fixes for thread local storage (TLS) in position independent executables (PIE).
  • Fixes to reproducible builds.
  • Fixed a performance regression in tmpfs.
  • DRM/KMS improvements.
  • bwfm(4) wireless driver for Broadcom FullMAC PCI and USB devices added.
  • Various sh(1) fixes.
  • mfii(4) SAS driver added.
  • hcpcd(8) updated to 7.2.2
  • httpd(8) updated.
FreeNAS as your Server OS

What if you could have a server OS that had built in RAID, NAS and SAN functionality, and could manage packages, containers and VMs in a GUI? What if that server OS was also free to download and install? Wouldn’t that be kind of awesome? Wouldn’t that be FreeNAS? FreeNAS is the world’s number one, open source storage OS, but it also comes equipped with all the jails, plugins, and VMs you need to run additional server-level services for things like email and web site hosting. File, Block, and even Object storage is all built-in and can be enabled with a few clicks. The ZFS file system scales to more drives than you could ever buy, with no limits for dataset sizes, snapshots, and restores. FreeNAS is also 100% FreeBSD. This is the OS used in the Netflix CDN, your PS4, and the basis for iOS. Set up a jail and get started downloading packages like Apache or NGINX for web hosting or Postfix for email service. Just released, our new TrueCommand management platform also streamlines alerts and enables multi-system monitoring.

Beastie Bits Feedback/Questions
  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Wed, 05 Jun 2019 20:15:00 -0700
300: The Big Three
FreeBSD 11.3-beta 1 is out, BSDCan 2019 recap, OpenIndiana 2019.04 is out, Overview of ZFS Pools in FreeNAS, why open source firmware is important for security, a new Opnsense release, wireguard on OpenBSD, and more. Headlines FreeBSD 11.3-b1 is out BSDCan 2019 Recap
  • We’re back from BSDCan and it was a packed week as always.
  • It started with bhyvecon on Tuesday. Meanwhile, Benedict spent the whole day in productive meetings: annual FreeBSD Foundation board meeting and FreeBSD Journal editorial board meeting.
  • On Wednesday, tutorials for BSDCan started as well as the FreeBSD Developer Summit. In the mornings, there were presentations in the big auditorium, while working groups about networking, failsafe bootcode, development web services, swap space management, and testing/CI were held. Friday had a similar format with an update from the FreeBSD core team and the “have, need, want” session for FreeBSD 13. In the afternoon, there were working groups about translation tools, package base, GSoC/Outreachy, or general hacking. Benedict held his Icinga tutorial in the afternoon with about 15 people attending. Devsummit presentation slides can be found on the wiki page and video recordings done by ScaleEngine are available on FreeBSD’s youtube channel.
  • The conference program was a good mixture of sysadmin and tech talks across the major BSDs. Benedict saw the following talks: How ZFS snapshots really work by Matt Ahrens, 20 years in Jail by Michael W. Lucas, OpenZFS BOF session, the future of OpenZFS and FreeBSD, MQTT for system administrators by Jan-Piet Mens, and spent the rest of the time in between in the hallway track.
  • Photos from the event are available on Ollivier Robert’s talegraph and Diane Bruce’s website for day 1, day 2, conference day 1, and conference day 2.
  • Thanks to all the sponsors, supporters, organizers, speakers, and attendees for making this yet another great BSDCan. Next year’s BSDCan will be from June 2 - 6, 2020.
OpenIndiana 2019.04 is out

We have released a new OpenIndiana Hipster snapshot 2019.04. The noticeable changes:

  • Firefox was updated to 60.6.3 ESR

  • Virtualbox packages were added (including guest additions)

  • Mate was updated to 1.22

  • IPS has received updates from OmniOS CE and Oracle IPS repos, including automatic boot environment naming

  • Some OI-specific applications have been ported from Python 2.7/GTK 2 to Python 3.5/GTK 3

  • Quick Demo Video: https://www.youtube.com/watch?v=tQ0-fo3XNrg

News Roundup Overview of ZFS Pools in FreeNAS

FreeNAS uses the OpenZFS (ZFS) file system, which handles both disk and volume management. ZFS offers RAID options mirror, stripe, and its own parity distribution called RAIDZ that functions like RAID5 on hardware RAID. The file system is extremely flexible and secure, with various drive combinations, checksums, snapshots, and replication all possible. For a deeper dive on ZFS technology, read the ZFS Primer section of the FreeNAS documentation.

SUGGEST LAYOUT attempts to balance usable capacity and redundancy by automatically choosing an ideal vdev layout for the number of available disks.

  • The following vdev layout options are available when creating a pool:
    • Stripe data is shared on two drives, similar to RAID0)
    • Mirror copies data on two drives, similar to RAID1 but not limited to 2 disks)
    • RAIDZ1 single parity similar to RAID5
    • RAIDZ2 double parity similar to RAID6
    • RAIDZ3 which uses triple parity and has no RAID equivalent
Why OpenSource Firmware is Important for Security
  • Roots of Trust

The goal of the root of trust should be to verify that the software installed in every component of the hardware is the software that was intended. This way you can know without a doubt and verify if hardware has been hacked. Since we have very little to no visibility into the code running in a lot of places in our hardware it is hard to do this. How do we really know that the firmware in a component is not vulnerable or that is doesn’t have any backdoors? Well we can’t. Not unless it was all open source. Every cloud and vendor seems to have their own way of doing a root of trust. Microsoft has Cerberus, Google has Titan, and Amazon has Nitro. These seem to assume an explicit amount of trust in the proprietary code (the code we cannot see). This leaves me with not a great feeling. Wouldn’t it be better to be able to use all open source code? Then we could verify without a doubt that the code you can read and build yourself is the same code running on hardware for all the various places we have firmware. We could then verify that a machine was in a correct state without a doubt of it being vulnerable or with a backdoor. It makes me wonder what the smaller cloud providers like DigitalOcean or Packet have for a root of trust. Often times we only hear of these projects from the big three or five.

OPNsense

This update addresses several privilege escalation issues in the access control implementation and new memory disclosure issues in Intel CPUs. We would like to thank Arnaud Cordier and Bill Marquette for the top-notch reports and coordination.

  • Here are the full patch notes:

  • system: address CVE-2019-11816 privilege escalation bugs[1] (reported by Arnaud Cordier)

  • system: /etc/hosts generation without interfacehasgateway()

  • system: show correct timestamp in config restore save message (contributed by nhirokinet)

  • system: list the commands for the pluginctl utility when n+ argument is given

  • system: introduce and use userIsAdmin() helper function instead of checking for 'page-all' privilege directly

  • system: use absolute path in widget ACLs (reported by Netgate)

  • system: RRD-related cleanups for less code exposure

  • interfaces: add EN DUID Generation using OPNsense PEN (contributed by Team Rebellion)

  • interfaces: replace legacygetallinterface_addresses() usage

  • firewall: fix port validation in aliases with leading / trailing spaces

  • firewall: fix outbound NAT translation display in overview page

  • firewall: prevent CARP outgoing packets from using the configured gateway

  • firewall: use CARP net.inet.carp.demotion to control current demotion in status page

  • firewall: stop live log poller on error result

  • dhcpd: change rule priority to 1 to avoid bogon clash

  • dnsmasq: only admins may edit custom options field

  • firmware: use insecure mode for base and kernel sets when package fingerprints are disabled

  • firmware: add optional device support for base and kernel sets

  • firmware: add Hostcentral mirror (HTTP, Melbourne, Australia)

  • ipsec: always reset rightallowany to default when writing configuration

  • lang: say "hola" to Spanish as the newest available GUI language

  • lang: updates for Chinese, Czech, Japanese, German, French, Russian and Portuguese

  • network time: only admins may edit custom options field

  • openvpn: call openvpnrefreshcrls() indirectly via plugin_configure() for less code exposure

  • openvpn: only admins may edit custom options field to prevent privilege escalation (reported by Bill Marquette)

  • openvpn: remove custom options field from wizard

  • unbound: only admins may edit custom options field

  • wizard: translate typehint as well

  • plugins: os-freeradius 1.9.3 fixes string interpolation in LDAP filters (contributed by theq86)

  • plugins: os-nginx 1.12[2]

  • plugins: os-theme-cicada 1.17 (contributed by Team Rebellion)

  • plugins: os-theme-tukan 1.17 (contributed by Team Rebellion)

  • src: timezone database information update[3]

  • src: install(1) broken with partially matching relative paths[4]

  • src: microarchitectural Data Sampling (MDS) mitigation[5]

  • ports: carootnss 3.44

  • ports: php 7.2.18[6]

  • ports: sqlite 3.28.0[7]

  • ports: strongswan custom XAuth generic patch removed

wiregaurd on OpenBSD

Earlier this week I imported a port for WireGuard into the OpenBSD ports tree. At the moment we have the userland daemon and the tools available. The in-kernel implementation is only available for Linux. At the time of writing there are packages available for -current. Jason A. Donenfeld (WireGuard author) has worked to support OpenBSD in WireGuard and as such his post on ports@ last year got me interested in WireGuard, since then others have toyed with WireGuard on OpenBSD before and as such I've used Ted's article as a reference. Note however that some of the options mentioned there are no longer valid. Also, I'll be using two OpenBSD peers here. The setup will be as follows: two OpenBSD peers, of which we'll dub wg1 the server and wg2 the client. The WireGuard service on wg1 is listening on 100.64.4.3:51820.

  • Conclusion

WireGuard (cl)aims to be easier to setup and faster than OpenVPN and while I haven't been able to verify the latter, the first is certainly true...once you've figured it out. Most documentation out there is for Linux so I had to figure out the wireguardgo service and the tun parameters. But all in all, sure, it's easier. Especially the client configuration on iOS which I didn't cover here because it's essentially pkgadd libqrencode ; cat client.conf | qrencode -t ansiutf8, scan the code with the WireGuard app and you're good to go. What is particularly neat is that WireGuard on iOS supports Always-on.

Beastie Bits Feedback/Questions
  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Thu, 30 May 2019 09:00:00 -0700
299: The NAS Fleet
Running AIX on QEMU on Linux on Windows, your NAS fleet with TrueCommand, Unleashed 1.3 is available, LLDB: CPU register inspection support extension, V7 Unix programs often not written as expected, and more. Headlines Running AiX on QEMU on Linux on Windows

YES it’s real! I’m using the Linux subsystem on Windows, as it’s easier to build this Qemu tree from source. I’m using Debian, but these steps will work on other systems that use Debian as a base. first thing first, you need to get your system with the needed pre-requisites to compile Great with those in place, now clone Artyom Tarasenko’s source repository Since the frame buffer apparently isn’t quite working just yet, I configure for something more like a text mode build. Now for me, GCC 7 didn’t build the source cleanly. I had to make a change to the file config-host.mak and remove all references to -Werror. Also I removed the sound hooks, as we won’t need them. Now you can build Qemu. Okay, all being well you now have a Qemu. Now following the steps from Artyom Tarasenko’s blog post, we can get started on the install!

  • See article for rest of walkthrough.
Take Command of Your NAS Fleet with TrueCommand

Hundreds of thousands of FreeNAS and TrueNAS systems are deployed around the world, with many sites having dozens of systems. Managing multiple systems individually can be time-consuming. iXsystems has responded to the challenge by creating a “single pane of glass” application to simplify the scaling of data, drive management, and administration of iXsystems NAS platforms. We are proud to introduce TrueCommand. TrueCommand is a ZFS-aware management application that manages TrueNAS and FreeNAS systems. The public Beta of TrueCommand is available for download now. TrueCommand can be used with small iXsystems NAS fleets for free. Licenses can be purchased for large-scale deployments and enterprise support. TrueCommand expands on the ease of use and power of TrueNAS and FreeNAS systems with multi-system management and reporting.

News Roundup Unleashed 1.3 Released

This is the fourth release of Unleashed - an operating system fork of illumos. For more information about Unleashed itself and the download links, see our website. As one might expect, this release removes a few things. The most notable being the removal of ksh93 along with all its libs. As far as libc interfaces are concerned, a number of non-standard functions were removed. In general, they have been replaced by the standards-compliant versions. (getgrentr, fgetgrentr, getgrgidr, getgrnamr, ttynamer, getloginr, shmdt, sigwait, gethostname, putmsg, putpmsg, and getaddrinfo) Additionally, wordexp and wordfree have been removed from libc. Even though they are technically required by POSIX, software doesn't seem to use them. Because of the fragile implementation (shelling out), we took the OpenBSD approach and just removed them. The default compilation environment now includes XOPENSOURCE=700 and EXTENSIONS. Additionally, all applications now use 64-bit file offsets, making use of LARGEFILESOURCE, LARGEFILE64SOURCE, and FILEOFFSET_BITS unnecessary. Last but not least, nightly.sh is no more. In short, to build one simply runs 'make'. (See README for detailed build instructions.)

Why did we decide to fork illumos? After all, there are already many illumos distributions available to choose from. We felt we could do better than any of them by taking a more aggressive stance toward compatibility and reducing cruft from code and community interactions alike.

LLDB: extending CPU register inspection support

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support and updating NetBSD distribution to LLVM 8 (which is still stalled by unresolved regressions in inline assembly syntax). You can read more about that in my Mar 2019 report. In April, my main focus was on fixing and enhancing the support for reading and writing CPU registers. In this report, I'd like to shortly summarize what I have done, what I have learned in the process and what I still need to do.

  • Future plans

My work continues with the two milestones from last month, plus a third that's closely related: Add support for FPU registers support for NetBSD/i386 and NetBSD/amd64. Support XSAVE, XSAVEOPT, ... registers in core(5) files on NetBSD/amd64. Add support for Debug Registers support for NetBSD/i386 and NetBSD/amd64. The most important point right now is deciding on the format for passing the remaining registers, and implementing the missing ptrace interface kernel-side. The support for core files should follow using the same format then. Userland-side, I will work on adding matching ATF tests for ptrace features and implement LLDB side of support for the new ptrace interface and core file notes. Afterwards, I will start working on improving support for the same things on 32-bit (i386) executables.

V7 Unix programs are often not written the way you would expect

Yesterday I wrote that V7 ed read its terminal input in cooked mode a line at a time, which was an efficient, low-CPU design that was important on V7's small and low-power hardware. Then in comments, frankg pointed out that I was wrong about part of that, namely about how ed read its input.

  • Sidebar: An interesting undocumented ed feature

Reading this section of the source code for ed taught me that it has an interesting, undocumented, and entirely characteristic little behavior. Officially, ed commands that have you enter new text have that new text terminate by a . on a line by itself:

In other words, it turns a single line with '.' into an EOF. The consequence of this is that if you type a real EOF at the start of a line, you get the same result, thus saving you one character (you use Control-D instead of '.' plus newline). This is very V7 Unix behavior, including the lack of documentation.

This is also a natural behavior in one sense. A proper program has to react to EOF here in some way, and it might as well do so by ending the input mode. It's also natural to go on to try reading from the terminal again for subsequent commands; if this was a real and persistent EOF, for example because the pty closed, you'll just get EOF again and eventually quit. V7 ed is slightly unusual here in that it deliberately converts '.' by itself to EOF, instead of signaling this in a different way, but in a way that's also the simplest approach; if you have to have some signal for each case and you're going to treat them the same, you might as well have the same signal for both cases.

Modern versions of ed appear to faithfully reimplement this convenient behavior, although they don't appear to document it. I haven't checked OpenBSD, but both FreeBSD ed and GNU ed work like this in a quick test. I haven't checked their source code to see if they implement it the same way.

Beastie Bits Feedback/Questions
  • DJ - Feedback
  • Fabian - ZFS ARC
  • Caleb - Question
  • A small programming note: After BSDNow episode 300, the podcast will switch to audio-only, using a new higher quality recording and production system. The live stream will likely still include video.
  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Wed, 22 May 2019 11:00:00 -0700
298: BSD On The Road
36 year old UFS bug fixed, a BSD for the road, automatic upgrades with OpenBSD, DTrace ext2fs support in FreeBSD, Dedicated SSH tunnel user, upgrading VMM VMs to OpenBSD 6.5, and more. Headlines 36+ year old bug in FFS/UFS discovered and patched

This update eliminates a kernel stack disclosure bug in UFS/FFS directory entries that is caused by uninitialized directory entry padding written to the disk.

  • When the directory entry is written to disk, it is written as a full 32bit entry, and the unused bytes were not initialized, so could possibly contain sensitive data from the kernel stack It can be viewed by any user with read access to that directory. Up to 3 bytes of kernel stack are disclosed per file entry, depending on the the amount of padding the kernel needs to pad out the entry to a 32 bit boundary. The offset in the kernel stack that is disclosed is a function of the filename size. Furthermore, if the user can create files in a directory, this 3 byte window can be expanded 3 bytes at a time to a 254 byte window with 75% of the data in that window exposed. The additional exposure is done by removing the entry, creating a new entry with a 4-byte longer name, extracting 3 more bytes by reading the directory, and repeating until a 252 byte name is created. This exploit works in part because the area of the kernel stack that is being disclosed is in an area that typically doesn't change that often (perhaps a few times a second on a lightly loaded system), and these file creates and unlinks themselves don't overwrite the area of kernel stack being disclosed. It appears that this bug originated with the creation of the Fast File System in 4.1b-BSD (Circa 1982, more than 36 years ago!), and is likely present in every Unix or Unix-like system that uses UFS/FFS. Amazingly, nobody noticed until now. This update also adds the -z flag to fsck_ffs to have it scrub the leaked information in the name padding of existing directories. It only needs to be run once on each UFS/FFS filesystem after a patched kernel is installed and running. Submitted by: David G. Lawrence dg@dglawrence.com
  • So a patched kernel will no longer leak this data, and running the fsck_ffs -z command will erase any leaked data that may exist on your system
  • OpenBSD commit with additional detail on mitigations The impact on OpenBSD is very limited: 1 - such stack bytes can be found in raw-device reads, from group operator. If you can read the raw disks you can undertake other more powerful actions. 2 - read(2) upon directory fd was disabled July 1997 because I didn't like how grep * would display garbage and mess up the tty, and applying vis(3) for just directory reads seemed silly. read(2) was changed to return 0 (EOF). Sep 2016 this was further changed to EISDIR, so you still cannot see the bad bytes. 3 - In 2013 when guenther adapted the getdents(2) directory-reading system call to 64-bit ino_t, the userland data format changed to 8-byte-alignment, making it incompatible with the 4-byte-alignment UFS on-disk format. As a result of code refactoring the bad bytes were not copied to userland. Bad bytes will remain in old directories on old filesystems, but nothing makes those bytes user visible. There will be no errata or syspatch issued. I urge other systems which do expose the information to userland to issue errata quickly, since this is a 254 byte infoleak of the stack which is great for ROP-chain building to attack some other bug. Especially if the kernel has no layout/link-order randomization ...
NomadBSD, a BSD for the Road

As regular It’s FOSS readers should know, I like diving into the world of BSDs. Recently, I came across an interesting BSD that is designed to live on a thumb drive. Let’s take a look at NomadBSD. NomadBSD is different than most available BSDs. NomadBSD is a live system based on FreeBSD. It comes with automatic hardware detection and an initial config tool. NomadBSD is designed to “be used as a desktop system that works out of the box, but can also be used for data recovery, for educational purposes, or to test FreeBSD’s hardware compatibility.” This German BSD comes with an OpenBox-based desktop with the Plank application dock. NomadBSD makes use of the DSB project. DSB stands for “Desktop Suite (for) (Free)BSD” and consists of a collection of programs designed to create a simple and working environment without needing a ton of dependencies to use one tool. DSB is created by Marcel Kaiser one of the lead devs of NomadBSD. Just like the original BSD projects, you can contact the NomadBSD developers via a mailing list.

  • Version 1.2 Released

NomadBSD recently released version 1.2 on April 21, 2019. This means that NomadBSD is now based on FreeBSD 12.0-p3. TRIM is now enabled by default. One of the biggest changes is that the initial command-line setup was replaced with a Qt graphical interface. They also added a Qt5 tool to install NomadBSD to your hard drive. A number of fixes were included to improve graphics support. They also added support for creating 32-bit images.

  • Thoughts on NomadBSD

I first discovered NomadBSD back in January when they released 1.2-RC1. At the time, I had been unable to install Project Trident on my laptop and was very frustrated with BSDs. I downloaded NomadBSD and tried it out. I initially ran into issues reaching the desktop, but RC2 fixed that issue. However, I was unable to get on the internet, even though I had an Ethernet cable plugged in. Luckily, I found the wifi manager in the menu and was able to connect to my wifi. Overall, my experience with NomadBSD was pleasant. Once I figured out a few things, I was good to go. I hope that NomadBSD is the first of a new generation of BSDs that focus on mobility and ease of use. BSD has conquered the server world, it’s about time they figured out how to be more user-friendly.

News Roundup [OpenBSD automatic

upgrade](https://www.tumfatig.net/20190426/openbsd-automatic-upgrade/)

OpenBSD 6.5 advertises for an installer improvement: rdsetroot(8) (a build-time tool) is now available for general use. Used in combination with autoinstall.8, it is now really easy to do automatic upgrades of your OpenBSD instances. I first manually upgraded my OpenBSD sandbox to 6.5. Once that was done, I could use the stock rdsetroot(8) tool. The plan is quite simple: write an unattended installation response file, insert it to a bsd.rd 6.5 installation image and reboot my other OpenBSD instances using that image.

  • Extra notes

There must be a way to run onetime commands (in the manner of fw_update) to automatically run sysmerge and packages upgrades. As for now, I’d rather do it manually. This worked like a charm on two Synology KVM instances using a single sd0 disk, on my Thinkpad X260 using Encrypted root with Keydisk and on a Vultr instance using Encrypted root with passphrase. And BTW, the upgrade on the X260 used the (iwn0) wireless connection. I just read that florian@ has released the sysupgrade(8) utility which should be released with OpenBSD 6.6. That will make upgrades even easier! Until then, happy upgrading.

FreeBSD Dtrace ext2fs Support
  • Which logs were replaced by dtrace-probes:

    • Misc printf's under DEBUG macro in the blocks allocation path.
    • Different on-disk structures validation errors, now the filesystem will silently return EIO's.
    • Misc checksum errors, same as above.
  • The only debug macro, which was leaved is EXT2FSPRINTEXTENTS.

  • It is impossible to replace it by dtrace-probes, because the additional logic is required to walk thru file extents.

  • The user still be able to see mount errors in the dmesg in case of:

    • Filesystem features incompatibility.
    • Superblock checksum error.

Create a dedicated user for ssh tunneling only

I use ssh tunneling A LOT, for everything. Yesterday, I removed the public access of my IMAP server, it’s now only available through ssh tunneling to access the daemon listening on localhost. I have plenty of daemons listening only on localhost that I can only reach through a ssh tunnel. If you don’t want to bother with ssh and redirect ports you need, you can also make a VPN (using ssh, openvpn, iked, tinc…) between your system and your server. I tend to avoid setting up VPN for the current use case as it requires more work and more maintenance than running ssh server and a ssh client. The last change, for my IMAP server, added an issue. I want my phone to access the IMAP server but I don’t want to connect to my main account from my phone for security reasons. So, I need a dedicated user that will only be allowed to forward ports. This is done very easily on OpenBSD. The steps are: 1. generate ssh keys for the new user 2. add an user with no password 3. allow public key for port forwarding Obviously, you must allow users (or only this one) to make port forwarding in your sshd_config.

That was easy. Some info on upgrading VMM VMs to 6.5

We're running dedicated vmm(4)/vmd(8) servers to host opinionated VMs. OpenBSD 6.5 is released! There are two ways you can upgrade your VM. Either do a manual upgrade or leverage autoinstall(8). You can take care of it via the console with vmctl(8).

  • Upgrade yourself

To get connected to the console you need to have access to the host your VM is running on. The same username and public SSH key, as provided for the VM, are used to create a local user on the host. When this is done you can use vmctl(8) to manage your VM. The options you have are:

```$ vmctl start id [-c]```

$ vmctl stop id [-fw]```

```-w Wait until the VM has been terminated.```

-c Automatically connect to the VM console.```

  • See the Article for the rest of the guide
Beastie Bits Feedback/Questions
  • Quentin - Organize an Ada/BSD interview
  • DJ - Update
  • Patrick - Bhyve frontends
  • A small programming note: After BSDNow episode 300, the podcast will switch to audio-only, using a new higher quality recording and production system. The live stream will likely still include video.
  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Wed, 15 May 2019 20:00:00 -0700
297: Dragonfly In The Wild
FreeBSD ZFS vs. ZoL performance, Dragonfly 5.4.2 has been release, containing web services with iocell, Solaris 11.4 SRU8, Problem with SSH Agent forwarding, OpenBSD 6.4 to 6.5 upgrade guide, and more. Headlines FreeBSD ZFS vs. ZoL Performance, Ubuntu ZFS On Linux Reference

With iX Systems having released new images of FreeBSD reworked with their ZFS On Linux code that is in development to ultimately replace their existing FreeBSD ZFS support derived from the code originally found in the Illumos source tree, here are some fresh benchmarks looking at the FreeBSD 12 performance of ZFS vs. ZoL vs. UFS and compared to Ubuntu Linux on the same system with EXT4 and ZFS. Using an Intel Xeon E3-1275 v6 with ASUS P10S-M WS motherboard, 2 x 8GB DDR4-2400 ECC UDIMMs, and Samsung 970 EVO Plus 500GB NVMe solid-state drive was used for all of this round of testing. Just a single modern NVMe SSD was used for this round of ZFS testing while as the FreeBSD ZoL code matures I'll test on multiple systems using a more diverse range of storage devices. FreeBSD 12 ZoL was tested using the iX Systems image and then fresh installs done of FreeBSD 12.0-RELEASE when defaulting to the existing ZFS root file-system support and again when using the aging UFS file-system. Ubuntu 18.04.2 LTS with the Linux 4.18 kernel was used when testing its default EXT4 file-system and then again when using the Ubuntu-ZFS ZoL support. Via the Phoronix Test Suite various BSD/Linux I/O benchmarks were carried out. Overall, the FreeBSD ZFS On Linux port is looking good so far and we are looking forward to it hopefully maturing in time for FreeBSD 13.0. Nice job to iX Systems and all of those involved, especially the ZFS On Linux project. Those wanting to help in testing can try the FreeBSD ZoL spins. Stay tuned for more benchmarks and on more diverse hardware as time allows and the FreeBSD ZoL support further matures, but so far at least the performance numbers are in good shape.

DragonFlyBSD 5.4.2 is out

Upgrading guide

Here's the tag commit, for what has changed from 5.4.1 to 5.4.2 The normal ISO and IMG files are available for download and install, plus an uncompressed ISO image for those installing remotely. I uploaded them to mirror-master.dragonflybsd.org last night so they should be at your local mirror or will be soon. This version includes Matt's fix for the HAMMER2 corruption bug he identified recently. If you have an existing 5.4 system and are running a generic kernel, the normal upgrade process will work.

> cd /usr/src > git pull > make buildworld. > make buildkernel. > make installkernel. > make installworld > make upgrade

After your next reboot, you can optionally update your rescue system:

> cd /usr/src > make initrd

As always, make sure your packages are up to date:

> pkg update > pkg upgrade News Roundup Containing web services with iocell

I'm a huge fan of the FreeBSD jails feature. It is a great system for splitting services into logical units with all the performance of the bare metal system. In fact, this very site runs in its own jail! If this is starting to sound like LXC or Docker, it might surprise you to learn that OS-level virtualization has existed for quite some time. Kudos to the Linux folks for finally getting around to it. 😛 If you're interested in the history behind Jails, there is an excellent talk from Papers We Love on the subject: https://www.youtube.com/watch?v=hgN8pCMLI2U

  • Getting started

There are plenty of options when it comes to setting up the jail system. Ezjail and Iocage seem popular, or you could do things manually. Iocage was recently rewritten in python, but was originally a set of shell scripts. That version has since been forked under the name Iocell, and I think it's pretty neat, so this tutorial will be using Iocell.

  • To start, you'll need the following:
    • A FreeBSD install (we'll be using 11.0)
    • The iocell package (available as a package, also in the ports tree)
    • A ZFS pool for hosting the jails

Once you have installed iocell and configured your ZFS pool, you'll need to run a few commands before creating your first jail. First, tell iocell which ZFS pool to use by issuing iocell activate $POOLNAME. Iocell will create a few datasets.

As you can imagine, your jails are contained within the /iocell/jails dataset. The /iocell/releases dataset is used for storing the next command we need to run, iocell fetch. Iocell will ask you which release you'd like to pull down. Since we're running 11.0 on the host, pick 11.0-RELEASE. Iocell will download the necessary txz files and unpack them in /iocell/releases.

  • See Article for the rest of the walkthrough.
Oracle Solaris 11.4 SRU8

Today we are releasing the SRU 8 for Oracle Solaris 11.4. It is available via 'pkg update' from the support repository or by downloading the SRU from My Oracle Support Doc ID 2433412.1.

  • This SRU introduces the following enhancements:
    • Integration of 28060039 introduced an issue where any firmware update/query commands will log eereports and repeated execution of such commands led to faulty/degraded NIC. The issue has been addressed in this SRU.
    • UCB (libucb, librpcsoc, libdbm, libtermcap, and libcurses) libraries have been reinstated for Oracle Solaris 11.4
    • Re-introduction of the service fc-fabric.
    • ibus has been updated to 1.5.19
  • The following components have also been updated to address security issues:
    • NTP has been updated to 4.2.8p12
    • Firefox has been updated to 60.6.0esr
    • BIND has been updated to 9.11.6
    • OpenSSL has been updated to 1.0.2r
    • MySQL has been updated to 5.6.43 & 5.7.25
    • libxml2 has been updated to 2.9.9
    • libxslt has been updated to 1.1.33
    • Wireshark has been updated to 2.6.7
    • ncurses has been updated to 6.1.0.20190105
    • Apache Web Server has been updated to 2.4.38
    • perl 5.22
    • pkg.depot
The Problem with SSH Agent Forwarding

After hacking the matrix.org website today, the attacker opened a series of GitHub issues mentioning the flaws he discovered. In one of those issues, he mentions that “complete compromise could have been avoided if developers were prohibited from using [SSH agent forwarding].” Here’s what man ssh_config has to say about ForwardAgent: "Agent forwarding should be enabled with caution. Users with the ability to bypass file permissions on the remote host (for the agent’s Unix-domain socket) can access the local agent through the forwarded connection. An attacker cannot obtain key material from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into the agent."" Simply put: if your jump box is compromised and you use SSH agent forwarding to connect to another machine through it, then you risk also compromising the target machine! Instead, you should use either ProxyCommand or ProxyJump (added in OpenSSH 7.3). That way, ssh will forward the TCP connection to the target host via the jump box and the actual connection will be made on your workstation. If someone on the jump box tries to MITM your connection, then you will be warned by ssh.

[OpenBSD Upgrade Guide: 6.4 to 6.5

Start by performing the pre-upgrade steps. Next, boot from the install kernel, bsd.rd: use bootable install media, or place the 6.5 version of bsd.rd in the root of your filesystem and instruct the boot loader to boot this kernel. Once this kernel is booted, choose the (U)pgrade option and follow the prompts. Apply the configuration changes and remove the old files. Finish up by upgrading the packages: pkg_add -u. Alternatively, you can use the manual upgrade process. You may wish to check the errata page or upgrade to the stable branch to get any post-release fixes.

  • Before rebooting into the install kernel
  • Configuration and syntax changes
  • Files to remove
  • Special packages
  • Upgrade without the install kernel
Beastie Bits Feedback/Questions
  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Wed, 08 May 2019 21:00:00 -0700
296: It’s Alive: OpenBSD 6.5
OpenBSD 6.5 has been released, mount ZFS datasets anywhere, help test upcoming NetBSD 9 branch, LibreSSL 2.9.1 is available, Bail Bond Denied Edition of FreeBSD Mastery: Jails, and one reason ed(1) was a good editor back in the days in this week’s episode. Headlines OpenBSD 6.5 Released
  • Changelog
  • Mirrors
  • 6.5 Includes
    • OpenSMTPD 6.5.0
    • LibreSSL 2.9.1
    • OpenSSH 8.0
    • Mandoc 1.14.5
    • Xenocara
    • LLVM/Clang 7.0.1 (+ patches)
    • GCC 4.2.1 (+ patches) and 3.3.6 (+ patches)
  • Many pre-built packages for each architecture:
    • aarch64: 9654
    • amd64: 10602
    • i386: 10535
Mount your ZFS datasets anywhere you want

ZFS is very flexible about mountpoints, and there are many features available to provide great flexibility. When you create zpool maintank, the default mountpoint is /maintank. You might be happy with that, but you don’t have to be content. You can do magical things.

  • Some highlights are:
    • mount point can be inherited
    • not all filesystems in a zpool need to be mounted
    • each filesystem (directory) can have different ZFS characteristics
    • In my case, let’s look at this new zpool I created earlier today and I will show you some very simple alternatives. This zpool use NVMe devices which should be faster than SSDs especially when used with multiple concurrent writes. This is my plan: run all the Bacula regression tests concurrently.
News Roundup Branch for netbsd 9 upcoming, please help and test -current

Folks, once again we are quite late for branching the next NetBSD release (NetBSD 9). Initially planned to happen early in February 2019, we are now approaching May and it is unlikely that the branch will happen before that. On the positive side, lots of good things landed in -current in between, like new Mesa, new jemalloc, lots of ZFS improvements - and some of those would be hard to pull up to the branch later. On the bad side we saw lots of churn in -current recently, and there is quite some fallout where we not even have a good overview right now. And this is where you can help:

  • please test -current, on all the various machines you have
  • especially interesting would be test results from uncommon architectures or strange combinations (like the sparc userland on sparc64 kernel issue I ran in yesterday) Please test, report success, and file PRs for failures! We will likely announce the real branch date on quite short notice, the likely next candidates would be mid may or end of may. We may need to do extra steps after the branch (like switch some architectures back to old jemalloc on the branch). However, the less difference between -current and the branch, the easier will the release cycle go. Our goal is to have an unprecedented short release cycle this time. But.. we always say that upfront.
LibreSSL 2.9.1 Released

We have released LibreSSL 2.9.1, which will be arriving in the LibreSSL directory of your local OpenBSD mirror soon. This is the first stable release from the 2.9 series, which is also included with OpenBSD 6.5

It includes the following changes and improvements from LibreSSL 2.8.x:

  • API and Documentation Enhancements

    • CRYPTO_LOCK is now automatically initialized, with the legacy callbacks stubbed for compatibility.
    • Added the SM3 hash function from the Chinese standard GB/T 32905-2016.
    • Added the SM4 block cipher from the Chinese standard GB/T 32907-2016.
    • Added more OPENSSLNO* macros for compatibility with OpenSSL.
    • Partial port of the OpenSSL ECKEYMETHOD API for use by OpenSSH.
    • Implemented further missing OpenSSL 1.1 API.
    • Added support for XChaCha20 and XChaCha20-Poly1305.
    • Added support for AES key wrap constructions via the EVP interface.
  • Compatibility Changes

    • Added pbkdf2 key derivation support to openssl(1) enc.
    • Changed the default digest type of openssl(1) enc to sha256.
    • Changed the default digest type of openssl(1) dgst to sha256.
    • Changed the default digest type of openssl(1) x509 -fingerprint to sha256.
    • Changed the default digest type of openssl(1) crl -fingerprint to sha256.
  • Testing and Proactive Security

    • Added extensive interoperability tests between LibreSSL and OpenSSL 1.0 and 1.1.
    • Added additional Wycheproof tests and related bug fixes.
  • Internal Improvements

    • Simplified sigalgs option processing and handshake signing algorithm selection.
    • Added the ability to use the RSA PSS algorithm for handshake signatures.
    • Added bnrandinterval() and use it in code needing ranges of random bn values.
    • Added functionality to derive early, handshake, and application secrets as per RFC8446.
    • Added handshake state machine from RFC8446.
    • Removed some ASN.1 related code from libcrypto that had not been used since around 2000.
    • Unexported internal symbols and internalized more record layer structs.
    • Removed SHA224 based handshake signatures from consideration for use in a TLS 1.2 handshake.
  • Portable Improvements

    • Added support for assembly optimizations on 32-bit ARM ELF targets.
    • Added support for assembly optimizations on Mingw-w64 targets.
    • Improved Android compatibility
  • Bug Fixes

    • Improved protection against timing side channels in ECDSA signature generation.
    • Coordinate blinding was added to some elliptic curves. This is the last bit of the work by Brumley et al. to protect against the Portsmash vulnerability.
    • Ensure transcript handshake is always freed with TLS 1.2.

The LibreSSL project continues improvement of the codebase to reflect modern, safe programming practices. We welcome feedback and improvements from the broader community. Thanks to all of the contributors who helped make this release possible.

FreeBSD Mastery: Jails – Bail Bond Denied Edition

I had a brilliant, hideous idea: to produce a charity edition of FreeBSD Mastery: Jails featuring the cover art I would use if I was imprisoned and did not have access to a real cover artist. (Never mind that I wouldn’t be permitted to release books while in jail: we creative sorts scoff at mere legal and cultural details.) I originally wanted to produce my own take on the book’s cover art. My first attempt failed spectacularly. I downgraded my expectations and tried again. And again. And again. I’m pleased to reveal the final cover for FreeBSD Mastery: Jails–Bail Bond Edition! This cover represents the very pinnacle of my artistic talents, and is the result of literally hours of effort. But, as this book is available only to the winner of charity fund-raisers, purchase of this tome represents moral supremacy. I recommend flaunting it to your family, coworkers, and all those of lesser character. Get your copy by winning the BSDCan 2019 charity auction… or any other other auction-type event I deem worthwhile. As far as my moral fiber goes: I have learned that art is hard, and that artists are not paid enough. And if I am ever imprisoned, I do hope that you’ll contribute to my bail fund. Otherwise, you’ll get more covers like this one.

One reason ed(1) was a good editor back in the days of V7 Unix

It is common to describe ed(1) as being line oriented, as opposed to screen oriented editors like vi. This is completely accurate but it is perhaps not a complete enough description for today, because ed is line oriented in a way that is now uncommon. After all, you could say that your shell is line oriented too, and very few people use shells that work and feel the same way ed does. The surface difference between most people's shells and ed is that most people's shells have some version of cursor based interactive editing. The deeper difference is that this requires the shell to run in character by character TTY input mode, also called raw mode. By contrast, ed runs in what Unix usually calls cooked mode, where it reads whole lines from the kernel and the kernel handles things like backspace. All of ed's commands are designed so that they work in this line focused way (including being terminated by the end of the line), and as a whole ed's interface makes this whole line input approach natural. In fact I think ed makes it so natural that it's hard to think of things as being any other way. Ed was designed for line at a time input, not just to not be screen oriented. This input mode difference is not very important today, but in the days of V7 and serial terminals it made a real difference. In cooked mode, V7 ran very little code when you entered each character; almost everything was deferred until it could be processed in bulk by the kernel, and then handed to ed all in a single line which ed could also process all at once. A version of ed that tried to work in raw mode would have been much more resource intensive, even if it still operated on single lines at a time.

Beastie Bits Feedback/Questions
  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Fri, 03 May 2019 10:00:00 -0700
295: Fun with funlinkat()
Introducing funlinkat(), an OpenBSD Router with AT&T U-Verse, using NetBSD on a raspberry pi, ZFS encryption is still under development, Rump kernel servers and clients tutorial, Snort on OpenBSD 6.4, and more. Headlines Introducing funlinkat
  • It turns out, every file you have ever deleted on a unix machine was probably susceptible to a race condition

One of the first syscalls which was created in Unix-like systems is unlink. In FreeBSD this syscall is number 10 (source) and in Linux, the number is dependent on the architecture but for most of them is also the tenth syscall (source). This indicated that this is one of the primary syscalls. The unlink syscall is very simple and we provide one single path to the file that we want to remove. The “removing file” process itself is very interesting so let’s spend a moment to understand the it. First, by removing the file we are removing a link from the directory to it. In Unix-like systems we can have many links to a single file (hard links). When we remove all links to the file, the file system will mark the blocks used by the file as free (a different file system will behave differently but let’s not jump into a second digression). This is why the process is called unlinking and not “removing file”. While we unlink the file two or three things will happen:

  • We will remove an entry in the directory with the filename.
  • We will decrease a file reference count (in inode).
  • If links go to zero - the file will be removed from the disk (again this doesn't mean that the blocks from the disk will be filled with zeros, though this may happen depending on the file system and configuration. However, in most cases this means that the file system will mark those blocks to as free and use them to write new data later This mostly means that “removing file” from a directory is an operation on the directory and not on the file (inode) itself. Another interesting subject is what happens if our system will perform only first or second step from the list. This depends on the file system and this is also something we will leave for another time. The problem with the unlink and even unlinkat function is that we don’t have any guarantee of which file we really are unlinking.
    • When you delete a file using its name, you have no guarantee that someone has not already deleted the file, or renamed it, and created a new file with the name you are about to delete. We have some stats about the file that we want to unlink. We performed some tests. In the same time another process removed our file and recreated it. When we finally try to remove our file it is no longer the same file. It’s a classic race condition.
    • Many programs will perform checks before trying to remove a file, to make sure it is the correct file, that you have the correct permissions etc. However this exposes the ‘Time-of-Check / Time-of-Use’ class of bugs. I check if the file I am about to remove is the one I created yesterday, it is, so I call unlink() on it. However, between when I checked the date on the file, and when I call unlink, I, some program I am running, might have updated the file. Or a malicious user might have put some other file at that name, so I would be the one who deleted it. In Unix-like operating systems we can get a handle for our file called file - a descriptor. File descriptors guarantee us that all the operations that we will be performing on it are done on the same file (inode). Even if someone was to unlink a number of directories entries, the operating system will not free the structures behind the file descriptor, and we can detect the file that was removed by someone and recreated (or just unlinked). So, for example, we have an alternative functions fstat which allows us to get file status of the given descriptor We already know that the file may have many links on the disk which point to the single inode. What happens when we open the file? Simplifying: kernel creates a memory representation of the inode (the inode itself is stored on the disk) called vnode. This single representation is used by all processes to refer the inode to the disk. If in a process we open the same file (inode) using different names (for example through hard links) all those files will be linked to the single vnode. That means that the pathname is not stored in the kernel. This is basically the reason why we don’t have a funlink function so that instead of the path we are providing just the file descriptor to the file. If we performed the fdunlink syscall, the kernel wouldn’t know which directory entry you would like to remove. Another problem is more architectural: as we discussed earlier unlinking is really an operation on the directory not on the file (inode) itself, so using funlink(fd) may create some confusion because we are not removing the inode corresponding to the file descriptor, we are performing action on the directory which points to the file. After some discussion we decided that the only sensible option for FreeBSD would be to create a funlinkat() function. This syscall would only performs additional sanitary checks if we are removing a directory entry which corresponds to the inode stored which refers to the file descriptor. int funlinkat(int dfd, const char *path, int fd, int flags); The API above will check if the path opened relative to the dfd points to the same vnode. Thanks to that we removed a race condition because all those sanitary checks are performed in the kernel mode while the file system is locked and there is no possibility to change it. The fd parameter may be set to the FD_NONE value which will mean that the sanitary check should not be performed and funlinkat will behave just like unlinkat. As you can notice I often refer to the unlink syscall but at the end the APIs looks like unlinkat syscall. It is true that the unlink syscall is very old and kind of deprecated. That said I referred to unlink because it’s just simpler. These days unlink simply uses the same code as unlinkat.
Using an OpenBSD Router with AT&T U-Verse

I upgraded to AT&T's U-verse Gigabit internet service in 2017 and it came with an Arris BGW-210 as the WiFi AP and router. The BGW-210 is not a terrible device, but I already had my own Airport Extreme APs wired throughout my house and an OpenBSD router configured with various things, so I had no use for this device. It's also a potentially-insecure device that I can't upgrade or fully disable remote control over. Fully removing the BGW-210 is not possible as we'll see later, but it is possible to remove it from the routing path. This is how I did it with OpenBSD.

News Roundup How to use NetBSD on a Raspberry Pi

Do you have an old Raspberry Pi lying around gathering dust, maybe after a recent Pi upgrade? Are you curious about BSD Unix? If you answered "yes" to both of these questions, you'll be pleased to know that the first is the solution to the second, because you can run NetBSD, as far back as the very first release, on a Raspberry Pi. BSD is the Berkley Software Distribution of Unix. In fact, it's the only open source Unix with direct lineage back to the original source code written by Dennis Ritchie and Ken Thompson at Bell Labs. Other modern versions are either proprietary (such as AIX and Solaris) or clever re-implementations (such as Minix and GNU/Linux). If you're used to Linux, you'll feel mostly right at home with BSD, but there are plenty of new commands and conventions to discover. If you're still relatively new to open source, trying BSD is a good way to experience a traditional Unix. Admittedly, NetBSD isn't an operating system that's perfectly suited for the Pi. It's a minimal install compared to many Linux distributions designed specifically for the Pi, and not all components of recent Pi models are functional under NetBSD yet. However, it's arguably an ideal OS for the older Pi models, since it's lightweight and lovingly maintained. And if nothing else, it's a lot of fun for any die-hard Unix geek to experience another side of the POSIX world.

ZFS Encryption is still under development (as of March 2019)

One of the big upcoming features that a bunch of people are looking forward to in ZFS is natively encrypted filesystems. This is already in the main development tree of ZFS On Linux, will likely propagate to FreeBSD (since FreeBSD ZFS will be based on ZoL), and will make it to Illumos if the Illumos people want to pull it in. People are looking forward to native encryption so much, in fact, that some of them have started using it in ZFS On Linux already, using either the development tip or one of the 0.8.0 release candidate pre-releases (ZoL is up to 0.8.0-rc3 as of now). People either doing this or planning to do this show up on the ZoL mailing list every so often.

Tutorial On Rump Kernel Servers and Clients

The rump anykernel architecture allows to run highly componentized kernel code configurations in userspace processes. Coupled with the rump sysproxy facility it is possible to run loosely distributed client-server "mini-operating systems". Since there is minimum configuration and the bootstrap time is measured in milliseconds, these environments are very cheap to set up, use, and tear down on-demand. This document acts as a tutorial on how to configure and use unmodified NetBSD kernel drivers as userspace services with utilities available from the NetBSD base system. As part of this, it presents various use cases. One uses the kernel cryptographic disk driver (cgd) to encrypt a partition. Another one demonstrates how to operate an FFS server for editing the contents of a file system even though your user account does not have privileges to use the host's mount() system call. Additionally, using a userspace TCP/IP server with an unmodified web browser is detailed.

Installing Snort on OpenBSD 6.4

As you may recall from previous posts, I am running an OpenBSD server on an APU2 air-cooled 3 Intel NIC box as my router/firewall for my secure home network. Given that all of my Internet traffic flows through this box, I thought it would be a cool idea to run an Intrusion Detection System (IDS) on it. Snort is the big hog of the open source world so I took a peek in the packages directory on one of the mirrors and lo and behold we have the latest & greatest version of Snort available! Thanks devs!!! I did some quick Googling and didn’t find much “modern” howto help out there so, after some trial and error, I have it up and running. I thought I’d give back in a small way and share a quickie howto for other Googlers out there who are looking for guidance. Here’s hoping that my title is good enough “SEO” to get you here!

Beastie Bits Feedback/Questions
  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Thu, 25 Apr 2019 13:00:00 -0700
294: The SSH Tarpit
A PI-powered Plan 9 cluster, an SSH tarpit, rdist for when Ansible is too much, falling in love with OpenBSD again, how I created my first FreeBSD port, the Tilde Institute of OpenBSD education and more. Headlines A Pi-Powered Plan 9 Cluster

Plan 9 from Bell Labs comes from the same stable as the UNIX operating system, which of course Linux was designed after, and Apple’s OS X runs on top of a certified UNIX operating system. Just like UNIX, Plan 9 was developed as a research O/S — a vehicle for trying out new concepts — with it building on key UNIX principles and taking the idea of devices are just files even further. In this post, we take a quick look at the Plan 9 O/S and some of the notable features, before moving on to the construction of a self-contained 4-node Raspberry Pi cluster that will provide a compact platform for experimentation.

Endlessh: an SSH Tarpit

I’m a big fan of tarpits: a network service that intentionally inserts delays in its protocol, slowing down clients by forcing them to wait. This arrests the speed at which a bad actor can attack or probe the host system, and it ties up some of the attacker’s resources that might otherwise be spent attacking another host. When done well, a tarpit imposes more cost on the attacker than the defender. The Internet is a very hostile place, and anyone who’s ever stood up an Internet-facing IPv4 host has witnessed the immediate and continuous attacks against their server. I’ve maintained such a server for nearly six years now, and more than 99% of my incoming traffic has ill intent. One part of my defenses has been tarpits in various forms.

News Roundup rdist(1) – when Ansible is too much

The post written about rdist(1) on johan.huldtgren.com sparked us to write one as well. It's a great, underappreciated, tool. And we wanted to show how we wrapped doas(1) around it. There are two services in our infrastructure for which we were looking to keep the configuration in sync and to reload the process when the configuration had indeed changed. There is a pair of nsd(8)/unbound(8) hosts and a pair of hosts running relayd(8)/httpd(8) with carp(4) between them. We didn't have a requirement to go full configuration management with tools like Ansible or Salt Stack. And there wasn't any interest in building additional logic on top of rsync or repositories. > Enter rdist(1), rdist is a program to maintain identical copies of files over multiple hosts. It preserves the owner, group, mode, and mtime of files if possible and can update programs that are executing.

Falling in love with OpenBSD again

I was checking the other day and was appalled at how long it has been since I posted here. I had been working a job during 2018 that had me traveling 3,600 miles by air every week so that is at least a viable excuse. So what is my latest project? I wanted to get something better than the clunky old T500 “freedom laptop” that I could use as my daily driver. Some background here. My first paid gig as a programmer was on SunOS 4 (predecessor to Solaris) and Ultrix (on a DEC MicroVAX). I went from there to a Commodore Amiga (preemptive multitasking in 1985!). I went from there to OS/2 (I know, patron saint of lost causes) and then finally decided to “sell out” and move to Windows as the path of least resistance in the mid 90’s. My wife bought me an iPod literally just as they started working with computers other than Macs and I watched with fascination as Apple made the big gamble and moved away from PowerPC chips to Intel. That was the beginning of the Apple Fan Boi years for me. My gateway drug was a G4 MacMini and I managed somehow to get in on the pre-production, developer build of an Intel-based Mac. I was quite happy on the platform until about three years ago.

How I Created My First FreeBSD Port

I created my first FreeBSD port recently. I found that FreeBSD didn't have a port for GoCD, which is a continuous integration and continuous deployment (CI/CD) system. This was a great opportunity to learn how to build a FreeBSD port while also contributing back to the community

The Tilde Institute of OpenBSD Education

Welcome to tilde.institute! This is an OpenBSD machine whose purpose is to provide a space in the tildeverse for experimentation with and education of the OpenBSD operating system. A variety of editors, shells, and compilers are installed to allow for development in a native OpenBSD environment. OpenBSD's httpd(8) is configured with slowcgi(8) as the fastcgi provider and sqlite3 available. This allows users to experiment with web development using compiled CGI in C, aka the BCHS Stack. In addition to php7.0 and mysql (mariadb) by request, this provides an environment where the development of complex web apps is possible.

Beastie Bits Feedback/Questions
  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Thu, 18 Apr 2019 09:00:00 -0700
293: Booking Jails
This week we have a special episode with a Michael W. Lucas interview about his latest jail book that’s been released. We’re talking all things jails, writing, book sponsoring, the upcoming BSDCan 2019 conference, and more.

###Interview - Michael W. Lucas - mwl@mwl.io / @mwlauthor
FreeBSD Mastery: Jails

  • BR: Welcome back to the show and congratulations on your latest book. How many books did you have to write before you could start on FreeBSD Mastery: Jails?
  • AJ: How much research did you have to do about jails?
  • BR: The book talks about something called ‘incomplete’ jails. What do you mean by that?
  • AJ: There are a lot of jail management frameworks out there. Why did you chose to write about iocage in the book?
  • BR: How many jails do you run yourself?
  • AJ: Can you tell us a bit about how you handle book sponsorship these days?
  • BR: What other books (fiction and non-fiction) are you currently working on?
  • AJ: Which talks are you looking forward to attend at the upcoming BSDCan conference?
  • BR: How is the BSD user group going?
  • AJ: Anything else you’d like to mention before we release you from our interview jail cell?
  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Thu, 11 Apr 2019 09:00:00 -0700
292: AsiaBSDcon 2019 Recap
FreeBSD Q4 2018 status report, the GhostBSD alternative, the coolest 90s laptop, OpenSSH 8.0 with quantum computing resistant keys exchange, project trident: 18.12-U8 is here, and more.

##Headlines
###AsiaBSDcon 2019 recap

  • Both Allan and I attended AsiaBSDcon 2019 in Tokyo in mid march. After a couple of days of Tokyo sightseeing and tasting the local food, the conference started with tutorials.
  • Benedict gave his tutorial about “BSD-based Systems Monitoring with Icinga2 and OpenSSH”, while Allan ran the FreeBSD developer summit.
  • On the next day, Benedict attended the tutorial “writing (network) tests for FreeBSD” held by Kristof Provost. I learned a lot about Kyua, where tests live and how they are executed. I took some notes, which will likely become an article or chapter in the developers handbook about writing tests.
  • On the third day, Hiroki Sato officially opened the paper session and then people went into individual talks.
  • Benedict attended

    Adventure in DRMland - Or how to write a FreeBSD ARM64 DRM driver by Emmanuel
    Vadot

powerpc64 architecture support in FreeBSD ports by Piotr Kubaj
Managing System Images with ZFS by Allan Jude
FreeBSD - Improving block I/O compatibility in bhyve by Sergiu Weisz
Security Fantasies and Realities for the BSDs by George V.
Neville-Neil

ZRouter: Remote update of firmware by Hiroki Mori
Improving security of the FreeBSD boot process by Marcin Wojtas

  • Allan attended

    Adventures in DRMland by Emmanuel Vadot
    Intel HAXM by Kamil Rytarowski
    BSD Solutions in Australian NGOs
    Container Migration on FreeBSD by Yuhei Takagawa
    Security Fantasies and Realities for the BSDs by George Neville-Neil

ZRouter: Remote update of firmware by Hiroki Mori
Improving security of the FreeBSD boot process by Marcin Wojtas

  • When not in talks, time was spent in the hallway track and conversations would often continue over dinner.
  • Stay tuned for announcements about where AsiaBSDcon 2020 will be, as the Tokyo Olympics will likely force some changes for next year. Overall, it was nice to see people at the conference again, listen to talks, and enjoy the hospitality of Japan.

###FreeBSD Quarterly Status Report - Fourth Quarter 2018

Since we are still on this island among many in this vast ocean of the Internet, we write this message in a bottle to inform you of the work we have finished and what lies ahead of us. These deeds that we have wrought with our minds and hands, they are for all to partake of - in the hopes that anyone of their free will, will join us in making improvements. In todays message the following by no means complete or ordered set of improvements and additions will be covered:
i386 PAE Pagetables for up to 24GB memory support, Continuous Integration efforts, driver updates to ENA and graphics, ARM enhancements such as RochChip, Marvell 8K, and Broadcom support as well as more DTS files, more Capsicum possibilities, as well as pfsync improvements, and many more things that you can read about for yourselves.
Additionally, we bring news from some islands further down stream, namely the nosh project, HardenedBSD, ClonOS, and the Polish BSD User-Group.
We would, selfishly, encourage those of you who give us the good word to please send in your submissions sooner than just before the deadline, and also encourage anyone willing to share the good word to please read the section on which submissions we’re also interested in having.

###GhostBSD: A Solid Linux-Like Open Source Alternative

The subject of this week’s Linux Picks and Pans is a representative of a less well-known computing platform that coexists with Linux as an open source operating system. If you thought that the Linux kernel was the only open source engine for a free OS, think again. BSD (Berkeley Software Distribution) shares many of the same features that make Linux OSes viable alternatives to proprietary computing platforms.
GhostBSD is a user-friendly Linux-like desktop operating system based on TrueOS. TrueOS is, in turn, based on FreeBSD’s development branch. TrueOS’ goal is to combine the stability and security of FreeBSD with a preinstalled GNOME, MATE, Xfce, LXDE or Openbox graphical user interface.
I stumbled on TrueOS while checking out new desktop environments and features in recent new releases of a few obscure Linux distros. Along the way, I discovered that today’s BSD computing family is not the closed source Unix platform the “BSD” name might suggest.
In last week’s Redcore Linux review, I mentioned that the Lumina desktop environment was under development for an upcoming Redcore Linux release. Lumina is being developed primarily for BSD OSes. That led me to circle back to a review I wrote two years ago on Lumina being developed for Linux.
GhostBSD is a pleasant discovery. It has nothing to do with being spooky, either. That goes for both the distro and the open source computing family it exposes.
Keep reading to find out what piqued my excitement about Linux-like GhostBSD.

##News Roundup
SPARCbook 3000ST - The coolest 90s laptop

A few weeks back I managed to pick up an incredibly rare laptop in immaculate condition for $50 on Kijiji: a Tadpole Technologies SPARCbook 3000ST from 1997 (it also came with two other working Pentium laptops from the 1990s).
Sun computers were an expensive desire for many computer geeks in the 1990s, and running UNIX on a SPARC-based laptop was, well, just as cool as it gets. SPARC was an open hardware platform that anyone could make, and Tadpole licensed the Solaris UNIX operating system from Sun for their SPARCbooks. Tadpole essentially made high-end UNIX/VAX workstations on costly, unusual platforms (PowerPC, DEC Alpha, SPARC) but only their SPARCbooks were popular in the high-end UNIX market of the 1990s.

###OpenSSH 8.0 Releasing With Quantum Computing Resistant Keys

OpenSSH 7.9 came out with a host of bug fixes last year with few new features, as is to be expected in minor releases. However, recently, Damien Miller has announced that OpenSSH 8.0 is nearly ready to be released. Currently, it’s undergoing testing to ensure compatibility across supported systems.

Better Security
Copying filenames with scp will be more secure in OpenSSH 8.0 due to the fact that copying filenames from a remote to local directory will prompt scp to check if the files sent from the server match your request. Otherwise, an attack server would theoretically be able to intercept the request by serving malicious files in place of the ones originally requested. Knowing this, you’re probably better off never using scp anyway. OpenSSH advises against it:
“The scp protocol is outdated, inflexible and not readily fixed. We recommend the use of more modern protocols like sftp and rsync for file transfer instead.”

  • Interesting new features

ssh(1): When prompting whether to record a new host key, accept the key fingerprint as a synonym for “yes”. This allows the user to paste a fingerprint obtained out of band at the prompt and have the client do the comparison for you.

###Project Trident : 18.12-U8 Available

Thank you all for your patience! Project Trident has finally finished some significant infrastructure updates over the last 2 weeks, and we are pleased to announce that package update 8 for 18.12-RELEASE is now available.
To switch to the new update, you will need to open the “Configuration” tab in the update manager and switch to the new “Trident-release” package repository. You can also perform this transition via the command line by running: sudo sysup --change-train Trident-release

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Thu, 04 Apr 2019 08:00:00 -0700
291: Storage Changes Software
Storage changing software, what makes Unix special, what you need may be “pipeline +Unix commands”, running a bakery on Emacs and PostgreSQL, the ultimate guide to memorable tech talks, light-weight contexts, and more.

##Headlines

###Tracking a storage issue led to software change

Early last year we completed a massive migration that moved our customers’ hosting data off of a legacy datacenter (that we called FR-SD2) onto several new datacenters (that we call FR-SD3, FR-SD5, and FR-SD6) with much more modern, up-to-date infrastructure.
This migration required several changes in both the software and hardware we use, including switching the operating system on our storage units to FreeBSD.
Currently, we use the NFS protocol to provide storage and export the filesystems on Simple Hosting, our web hosting service, and the FreeBSD kernel includes an NFS server for just this purpose.

  • Problem

While migrating virtual disks of Simple Hosting instances from FR-SD2, we noticed high CPU load spikes on the new storage units.

###What Makes Unix Special

Ever since Unix burst onto the scene within the early '70s, observers within the pc world have been fast to put in writing it off as a unusual working system designed by and for knowledgeable programmers. Regardless of their proclamations, Unix refuses to die. Means again in 1985, Stewart Cheifet puzzled if Unix would turn out to be the usual working system of the longer term on the PBS present “The Laptop Chronicles,” though MS-DOS was effectively in its heyday. In 2018, it is clear that Unix actually is the usual working system, not on desktop PCs, however on smartphones and tablets.

  • What Makes Unix Special?

It is also the usual system for net servers. The actual fact is, hundreds of thousands of individuals all over the world have interacted with Linux and Unix programs daily, most of whom have by no means written a line of code of their lives.
So what makes Unix so beloved by programmers and different techie sorts? Let’s check out a few of issues this working system has going for it. (For some background on Unix, try The Historical past of Unix: From Bell Labs to the iPhone.)

##News Roundup
What you need may be “pipeline +Unix commands” only

I came across Taco Bell Programming recently, and think this article is worthy to read for every software engineer. The post mentions a scenario which you may consider to use Hadoop to solve but actually xargs may be a simpler and better choice. This reminds me a similar experience: last year a client wanted me to process a data file which has 5 million records. After some investigations, no novel technologies, a concise awk script (less than 10 lines) worked like a charm! What surprised me more is that awk is just a single-thread program, no nifty concurrency involved.
The IT field never lacks “new” technologies: cloud computing, big data, high concurrency, etc. However, the thinkings behind these “fancy” words may date back to the era when Unix arose. Unix command line tools are invaluable treasure. In many cases, picking the right components and using pipeline to glue them can satisfy your requirement perfectly. So spending some time in reviewing Unixcommand line manual instead of chasing state-of-the-art techniques exhaustedly, you may gain more.
BTW, if your data set can be disposed by an awk script, it should not be called “big data”.

###Running a bakery on Emacs and PostgreSQL

Just over a year ago now, I finally opened the bakery I’d been dreaming of for years. It’s been a big change in my life, from spending all my time sat in front of a computer, to spending most of it making actual stuff. And stuff that makes people happy, at that. It’s been a huge change, but I can’t think of a single job change that’s ever made me as happy as this one.
One of the big changes that came with going pro was that suddenly I was having to work out how much stuff I needed to mix to fill the orders I needed. On the face of it, this is really simple, just work out how much dough you need, then work out what quantities to mix to make that much dough. Easy. You can do it with a pencil and paper. Or, in traditional bakers’ fashion, by scrawling with your finger on a floured work bench.
And that’s how I coped for a few weeks early on. But I kept making mistakes, which makes for an inconsistent product (bread is very forgiving, you have to work quite hard to make something that isn’t bread, but consistency matters). I needed to automate.

###The Ultimate Guide To Memorable Tech Talks

Imagine this. You’re a woman in a male-dominated field. English is not your first language. Even though you’re confident in your engineering work, the thought of public speaking and being recorded for the world to see absolutely terrifies you.
That was me, five years ago. Since then, I’ve moved into a successful career in Developer Advocacy and spoken at dozens of technical events in the U.S. and worldwide.
I think everyone has the ability to deliver stellar conference talks, which is why I took the time to write this post.

  • The Ultimate Guide
  • 1: Introduction
  • 2: Choosing a Topic
  • 3: Writing a Conference Proposal (or CFP)
  • 4: Tools of the Trade
  • 5: Planning and Time Estimation
  • 6: Writing a Talk
  • 7: Practice and Delivery

###Light-weight Contexts: An OS Abstraction for Safety and Performance (2016)

Abstract: “We introduce a new OS abstraction—light-weight con-texts (lwCs)—that provides independent units of protection, privilege, and execution state within a process. A process may include several lwCs, each with possibly different views of memory, file descriptors, and access capabilities. lwCs can be used to efficiently implement roll-back (process can return to a prior recorded state),isolated address spaces (lwCs within the process may have different views of memory, e.g., isolating sensitive data from network-facing components or isolating different user sessions), and privilege separation (in-process reference monitors can arbitrate and control access).
lwCs can be implemented efficiently: the overhead of a lwC is proportional to the amount of memory exclusive to the lwC; switching lwCs is quicker than switching kernel threads within the same process. We describe the lwC abstraction and API, and an implementation of lwCs within the FreeBSD 11.0 kernel. Finally, we present an evaluation of common usage patterns, including fast roll-back, session isolation, sensitive data isolation, and in-process reference monitoring, using Apache, nginx, PHP,and OpenSSL.”

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Thu, 28 Mar 2019 07:00:00 -0700
290: Timestamped Notes
FreeBSD on Cavium ThunderX, looking at NetBSD as an OpenBSD user, taking time-stamped notes in vim, OpenBSD 6.5 has been tagged, FreeBSD and NetBSD in GSoC 2019, SecBSD: an UNIX-like OS for Hackers, and more.

##Headlines
###ARM’d and dangerous: FreeBSD on Cavium ThunderX (aarch64)

While I don’t remember for how many years I’ve had an interest in CPU architectures that could be an alternative to AMD64, I know pretty well when I started proposing to test 64-bit ARM at work. It was shortly after the disaster named Spectre / Meltdown that I first dug out server-class ARM hardware and asked whether we should get one such server and run some tests with it.
While the answer wasn’t a clear “no” it also wasn’t exactly “yes”. I tried again a few times over the course of 2018 and each time I presented some more points why I thought it might be a good thing to test this. But still I wasn’t able to get a positive answer. Finally in January 2019 year I got a definitive answer – and it was “yes, go ahead”! The fact that Amazon had just presented their Graviton ARM Processor may have helped the decision.

###Looking at NetBSD from an OpenBSD user perspective

I use to use NetBSD quite a lot. From 2.0 to 6.99. But for some reasons, I stopped using it about 2012, in favor of OpenBSD. Reading on the new 8 release, I wanted to see if all the things I didn’t like on NetBSD were gone. Here is a personal Pros / Cons list. No Troll, hopefully. Just trying to be objective.

  • What I liked (pros)
  • Things I didn’t like (cons)
  • Conclusion

So that was it. I didn’t spend more than 30 minutes of it. But I didn’t want to spend more time on it. I did stop using NetBSD because of the need to compile each and every packages ; it was in the early days of pkgin. I also didn’t like the way system maintenance was to be done. OpenBSD’s 6-months release seemed far more easy to manage. I still think NetBSD is a great OS. But I believe you have to spent more time on it than you would have to do with OpenBSD.
That said, I’ll keep using my Puffy OS.

##News Roundup
Using Vim to take time-stamped notes

I frequently find myself needing to take time-stamped notes. Specifically, I’ll be in a call, meeting, or interview and need to take notes that show how long it’s been since the meeting started.
My first thought was that there’s be a plugin to add time stamps, but a quick search didn’t turn anything up. However, I little digging did turn up the fact that vim has the built-in ability to tell time.
This means that writing a bit of vimscript to insert a time stamp is pretty easy. After a bit of fiddling, I came up with something that serves my needs, and I decided it might be useful enough to others to be worth sharing.

###OpenBSD 6.5-beta has been tagged

It’s that time of year again; Theo (deraadt@) has just tagged 6.5-beta. A good reminder for us all run an extra test install and see if your favorite port still works as you expect.

CVSROOT: /cvs
Module name: src
Changes by: deraadt@cvs.openbsd.org 2019/02/26 15:24:41

Modified files:
etc/root : root.mail
share/mk : sys.mk
sys/conf : newvers.sh
sys/sys : ktrace.h param.h
usr.bin/signify: signify.1
sys/arch/macppc/stand/tbxidata: bsd.tbxi

Log message:
crank to 6.5-beta

###The NetBSD Foundation participating in Google Summer of Code 2019

For the 4th year in a row and for the 13th time The NetBSD Foundation will participate in Google Summer of Code 2019!
If you are a student and would like to learn more about Google Summer of Code please go to the Google Summer of Code homepage.
You can find a list of projects in Google Summer of Code project proposals in the wiki.
Do not hesitate to get in touch with us via #netbsd-code IRC channel on Freenode and via NetBSD mailing lists!

###SecBSD: an UNIX-like OS for Hackers

SecBSD is an UNIX-like operating system focused on computer security based on OpenBSD. Designed for security testing, hacking and vulnerability assessment, it uses full disk encryption and ProtonVPN + OpenVPN by default.
A security BSD enviroment for security researchers, penetration testers, bug hunters and cybersecurity experts. Developed by Dark Intelligence Team for private use and will be public release coming soon.

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Thu, 21 Mar 2019 07:00:00 -0700
289: Microkernel Failure
A kernel of failure, IPv6 fragmentation vulnerability in OpenBSD’s pf, a guide to the terminal, using a Yubikey for SSH public key authentication, FreeBSD desktop series, and more.

##Headlines

###A Kernel Of Failure -
How IBM bet big on the microkernel being the next big thing in operating systems back in the ’90s—and spent billions with little to show for it.

Today in Tedium: In the early 1990s, we had no idea where the computer industry was going, what the next generation would look like, or even what the driving factor would be. All the developers back then knew is that the operating systems available in server rooms or on desktop computers simply weren’t good enough, and that the next generation needed to be better—a lot better. This was easier said than done, but this problem for some reason seemed to rack the brains of one company more than any other: IBM. Throughout the decade, the company was associated with more overwrought thinking about operating systems than any other, with little to show for it in the end. The problem? It might have gotten caught up in kernel madness. Today’s Tedium explains IBM’s odd operating system fixation, and the belly flops it created.

###CVE-2019-5597IPv6 fragmentation vulnerability in OpenBSD Packet Filter

Packet Filter is OpenBSD’s service for filtering network traffic and performing Network Address Translation. Packet Filter is also capable of normalizing and conditioning TCP/IP traffic, as well as providing bandwidth control and packet prioritization.
Packet Filter has been a part of the GENERIC kernel since OpenBSD 5.0.Because other BSD variants import part of OpenBSD code, Packet Filter is also shipped with at least the following distributions that are affected in a lesser extent: FreeBSD, pfSense, OPNSense, Solaris.

Note that other distributions may also contain Packet Filter but due to the imported version they might not be vulnerable. This advisory covers the latest OpenBSD’s Packet Filter. For specific details about other distributions, please refer to the advisory of the affected product.

##News Roundup
How I’m still not using GUIs in 2019: A guide to the terminal

TL;DR: Here are my dotfiles. Use them and have fun.

GUIs are bloatware. I’ve said it before. However, rather than just complaining about IDEs I’d like to provide an understandable guide to a much better alternative: the terminal.
IDE stands for Integrated Development Environment. This might be an accurate term, but when it comes to a real integrated development environment, the terminal is a lot better.
In this post, I’ll walk you through everything you need to start making your terminal a complete development environment: how to edit text efficiently, configure its appearance, run and combine a myriad of programs, and dynamically create, resize and close tabs and windows.

  • Don’t forget rule number one.

Whenever in doubt, read the manual.

###Using a Yubikey as smartcard for SSH public key authentication

SSH is an awesome tool. Logging into other machines securely is so pervasive to us sysadmins nowadays that few of us think about what’s going on underneath. Even more so once you start using the more advanced features such as the ssh-agent, agent-forwarding and ProxyJump. When doing so, care must be taken in order to not compromise one’s logins or ssh keys.
You might have heard of Yubikeys.
These are USB authentication devices that support several different modes: they can be used for OTP (One Time Password) authentication, they can store OpenPGP keys, be a 2-factor authentication token and they can act as a SmartCard.
In OpenBSD, you can use them for Login (with loginyubikey(8)) with OTP since 2012, and there are many descriptions available(1) how to set this up.

###The 18 Part FreeBSD Desktop Series by Vermaden

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Thu, 14 Mar 2019 16:00:00 -0700
288: Turing Complete Sed
Software will never fix Spectre-type bugs, a proof that sed is Turing complete, managed jails using Bastille, new version of netdata, using grep with /dev/null, using GMail with mutt, and more.

##Headlines
###Google: Software is never going to be able to fix Spectre-type bugs

Researchers from Google investigating the scope and impact of the Spectre attack have published a paper asserting that Spectre-like vulnerabilities are likely to be a continued feature of processors and, further, that software-based techniques for protecting against them will impose a high performance cost. And whatever the cost, the researchers continue, the software will be inadequate—some Spectre flaws don’t appear to have any effective software-based defense. As such, Spectre is going to be a continued feature of the computing landscape, with no straightforward resolution.
The discovery and development of the Meltdown and Spectre attacks was undoubtedly the big security story of 2018. First revealed last January, new variants and related discoveries were made throughout the rest of the year. Both attacks rely on discrepancies between the theoretical architectural behavior of a processor—the documented behavior that programmers depend on and write their programs against—and the real behavior of implementations.
Specifically, modern processors all perform speculative execution; they make assumptions about, for example, a value being read from memory or whether an if condition is true or false, and they allow their execution to run ahead based on these assumptions. If the assumptions are correct, the speculated results are kept; if it isn’t, the speculated results are discarded and the processor redoes the calculation. Speculative execution is not an architectural feature of the processor; it’s a feature of implementations, and so it’s supposed to be entirely invisible to running programs. When the processor discards the bad speculation, it should be as if the speculation never even happened.

###A proof that Unix utility sed is Turing complete

Many people are surprised when they hear that sed is Turing complete. How come a text filtering program is Turing complete, they wonder. Turns out sed is a tiny assembly language that has a comparison operation, a branching operation and a temporary buffer. These operations make sed Turing complete.
I first learned about this from Christophe Blaess. His proof is by construction – he wrote a Turing machine in sed (download turing.sed). As any programming language that can implement a Turing machine is Turing complete we must conclude that sed is also Turing complete.
Christophe offers his own introduction to Turing machines and a description of how his sed implementation works in his article Implementation of a Turing Machine as a sed Script.

Christophe isn’t the first person to realize that sed is almost a general purpose programming language. People have written tetris, sokoban and many other programs in sed. Take a look at these:

##News Roundup
Bastille helps you quickly create and manage FreeBSD Jails.

Bastille helps you quickly create and manage FreeBSD Jails.
Jails are extremely lightweight containers that provide a full-featured UNIX-like operating system inside. These containers can be used for software development, rapid testing, and secure production Internet services.
Bastille provides an interface to create, manage and destroy these secure virtualized environments.

###netdata v1.12 released

Netdata is distributed, real-time, performance and health monitoring for systems and applications. It is a highly optimized monitoring agent you install on all your systems and containers.
Netdata provides unparalleled insights, in real-time, of everything happening on the systems it runs (including web servers, databases, applications), using highly interactive web dashboards. It can run autonomously, without any third party components, or it can be integrated to existing monitoring tool chains (Prometheus, Graphite, OpenTSDB, Kafka, Grafana, etc).
Netdata is fast and efficient, designed to permanently run on all systems (physical & virtual servers, containers, IoT devices), without disrupting their core function.

  • Patch release 1.12.1 contains 22 bug fixes and 8 improvements.

###Using grep with /dev/null, an old Unix trick

Every so often I will find myself writing a grep invocation like this:

find .... -exec grep <something> /dev/null '{}' '+'

The peculiar presence of /dev/null here is an old Unix trick that is designed to force grep to always print out file names, even if your find only matches one file, by always insuring that grep has at least two files as arguments. You can wind up wanting to do the same thing with a direct use of grep if you’re not certain how many files your wildcard may match.

###USING GMAIL WITH MUTT

I recently switched to using mutt for email and while setting up mutt to use imap is pretty straightforward, this tutorial will also document some advanced concepts such as encrypting your account password and sending emails from a different From address.
This tutorial assumes that you have some familiarity with using mutt and have installed it with sidebar support (sudo apt-get install mutt-patched for the ubuntu folks) and are comfortable with editing your muttrc.
If you would just like to skip to the end, my mutt configuration file can be found here.

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Thu, 07 Mar 2019 07:00:00 -0800
287: rc.d in NetBSD
Design and Implementation of NetBSD’s rc.d system, first impressions of Project Trident 18.12, PXE booting a FreeBSD disk image, middle mouse button pasting, NetBSD gains hardware accelerated virtualization, and more.

##Headlines
###The Design and Implementation of the NetBSD rc.d system

  • Abstract

In this paper I cover the design and implementation of the rc.d system start-up mechanism in NetBSD 1.5, which replaced the monolithic /etc/rc start-up file inherited from 4.4BSD. Topics covered include a history of various UNIX start-up mechanisms (including NetBSD prior to 1.5), design considerations that evolved over six years of discussions, implementation details, an examination of the human issues that occurred during the design and implementation, as well as future directions for the system.

  • Introduction

NetBSD recently converted from the traditional 4.4BSD monolithic /etc/rc start-up script to an /etc/rc.d mechanism, where there is a separate script to manage each service or daemon, and these scripts are executed in a specific order at system boot.
This paper covers the motivation, design and implementation of the rc.d system; from the history of what NetBSD had before to the system that NetBSD 1.5 shipped with in December 2000, as well as future directions.
The changes were contentious and generated some of the liveliest discussions about any feature change ever made in NetBSD. Parts of those discussions will be covered to provide insight into some of the design and implementation decisions.

  • History

There is great diversity in the system start-up mechanisms used by various UNIX variants. A few of the more pertinent schemes are detailed below. As NetBSD is derived from 4.4BSD, it follows that a description of the latter’s method is relevant. Solaris’ start-up method is also detailed, as it is the most common System V UNIX variant.

###First impressions of Project Trident 18.12

Project Trident (hereafter referred to as Trident) is a desktop operating system based on TrueOS. Trident takes the rolling base platform of TrueOS, which is in turn based on FreeBSD’s development branch, and combines it with the Lumina desktop environment.

+Installing

The debut release of Trident is available as a 4.1GB download that can be burned to a disc or transferred to a USB thumb drive. Booting from the Trident media brings up a graphical interface and automatically launches the project’s system installer. Down the left side of the display there are buttons we can click to show hardware information and configuration options. These buttons let us know if our wireless card and video card are compatible with Trident and give us a chance to change our preferred language and keyboard layout. At the bottom of the screen we find buttons that will open a terminal or shutdown the computer.

  • Early impressions

Trident boots to a graphical login screen where we can sign into the Lumina desktop or a minimal Fluxbox session. Lumina, by default, uses Fluxbox as its window manager. The Lumina desktop places its panel along the bottom of the screen and an application menu sits in the bottom-left corner. On the desktop we find icons for opening the software manager, launching the Falkon web browser, running the VLC media player, opening the Control Panel and adjusting the Lumina theme.
The application menu has an unusual and compact layout. The menu shows just a search box and buttons for browsing applications, opening a file manager, accessing desktop settings and signing out. To see what applications are available we can click the Browse Applications entry, which opens a window in the menu where we can scroll through installed programs. This is a bit awkward since the display window is small and only shows a few items at a time.
Early on I found it is possible to swap out the default “Start menu” with an alternative “Application menu” through the Panels configuration tool. This alternative menu offers a classic tree-style application menu. I found the latter menu easier to navigate as it expands to show all the applications in a selected category.

  • Conclusions

I have a lot of mixed feelings and impressions when it comes to Trident. On the one hand, the operating system has some great technology under the hook. It has cutting edge packages from the FreeBSD ecosystem, we have easy access to ZFS, boot environments, and lots of open source packages. Hardware support, at least on my physical workstation, was solid and the Lumina desktop is flexible.

##News Roundup
PXE booting of a FreeBSD disk image

I had to set up a regression and network performance lab. This lab will be managed by a Jenkins, but the first step is to understand how to boot a FreeBSD disk by PXE. This article explains a simple way of doing it.
For information, all these steps were done using 2 PC Engines APU2 (upgraded with latest BIOS for iPXE support), so it’s a headless (serial port only, this can be IPMI SoL with different hardware) .

  • THE BIG PICTURE

Before explaining all steps and command line, here is the full big picture of the final process.

###Why I like middle mouse button paste in xterm so much

In my entry about how touchpads are not mice, I mused that one of the things I should do on my laptop was insure that I had a keyboard binding for paste, since middle mouse button is one of the harder multi-finger gestures to land on a touchpad. Kurt Mosiejczuk recently left a comment there where they said:
Shift-Insert is a keyboard equivalent for paste that is in default xterm (at least OpenBSD xterm, and putty on Windows too). I use that most of the time now as it seems less… trigger-happy than right click paste.
This sparked some thoughts, because I can’t imagine giving up middle mouse paste if I have a real choice. I had earlier seen shift-insert mentioned in other commentary on my entry and so have tried a bit to use it on my laptop, and it hasn’t really felt great even there; on my desktops, it’s even less appealing (I tried shift-insert out there to confirm that it did work in my set of wacky X resources).
In thinking about why this is, I came to the obvious realization about why all of this is so. I like middle mouse button paste in normal usage because it’s so convenient, because almost all of the time my hand is already on the mouse. And the reason my hand is already on the mouse is because I’ve just used the mouse to shift focus to the window I want to paste into. Even on my laptop, my right hand is usually away from the keyboard as I move the mouse pointer on the touchpad, making shift-Insert at least somewhat awkward.

###NetBSD Gains Hardware Accelerated Virtualization

  • NetBSD Virtual Machine Monitor

NVMM provides hardware-accelerated virtualization support for NetBSD. It is made of an ~MI frontend, to which MD backends can be plugged. A virtualization API is shipped via libnvmm, that allows to easily create and manage virtual machines via NVMM. Two additional components are shipped as demonstrators, toyvirt and smallkern: the former is a toy virtualizer, that executes in a VM the 64bit ELF binary given as argument, the latter is an example of such binary.

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Thu, 28 Feb 2019 09:00:00 -0800
286: Old Machine Revival
Adding glue to a desktop environment, flashing the BIOS on a PC Engine, revive a Cisco IDS into a capable OpenBSD computer, An OpenBSD WindowMaker desktop, RealTime data compression, the love for pipes, and more.

##Headlines
###Adding Glue To a Desktop Environment

In this article we will put some light on a lot of tools used in the world of Unix desktop environment customization, particularly regarding wmctrl, wmutils, xev, xtruss, xwininfo, xprop, xdotools, xdo, sxhkd, xbindkeys, speckeysd, xchainkeys, alttab, triggerhappy, gTile, gidmgr, keynav, and more. If those don’t make sense then this article will help. Let’s hope this can open your mind to new possibilities.
With that in mind we can wonder if what’s actually needed from a window manager, presentation and operation, can be split up and complemented with other tools. We can also start thinking laterally, the communication and interaction between the different components of the environment. We have the freedom to do so because the X protocol is transparent and components usually implement many standards for interfacing between windows. It’s like gluing parts together to create a desktop environment.

  • The tools we’ll talk about fall into one of those categories:
  • Debugging
  • Window manipulation
  • Simulation of interaction
  • Extended manipulation
  • Hotkey daemon
  • Layout manager

###Flashing the BIOS on the PC Engines APU4c4

I absolutely love the PC Engines APU devices. I use them for testing HardenedBSD experimental features in more constrained 64-bit environments and firewalls. Their USB and mSATA ports have a few quirks, and I bumped up against a major quirk that required flashing a different BIOS as a workaround. This article details the hacky way in which I went about doing that.
What prompted this article is that something in either the CAM or GEOM layer in FreeBSD 11.2 caused the mSATA to hang, preventing file writes. OPNsense 18.7 uses FreeBSD 11.1 whereas the recently-released OPNsense 19.1 uses HardenedBSD 11.2 (based on FreeBSD 11.2). I reached out to PC Engines directly, and they let me know that the issue is a known BIOS issue. Flashing the “legacy” BIOS series would provide me with a working system.
It also just so happens that a new “legacy” BIOS version was just released which turns on ECC mode for the RAM. So, I get a working OPNsense install AND ECC RAM! I’ll have one bird for dinner, the other for dessert.
Though I’m using an APU4, these instructions should work for the other APU devices. The BIOS ROM download URLs should be changed to reflect the device you’re targeting along with the BIOS version you wish to deploy.
SPECIAL NOTE: There be dragons! I’m primarily writing this article to document the procedure for my own purposes. My memory tends to be pretty faulty these days. So, if something goes wrong, please do not hold me responsible. You’re the one at the keyboard. ;)
VERY SPECIAL NOTE: We’ll use the mSATA drive for swap space, just in case. Should the swap space be used, it will destroy whatever is on the disk.

##News Roundup
Revive a Cisco IDS into a capable OpenBSD computer!

Even though Cisco equipment is very capable, it tends to become End-of-Life before you can say “planned obsolescence”. Websites become bigger, bandwidths increase, and as a side effect of those “improvements”, routers, firewalls, and in this case, intrusion prevention systems get old quicker and quicker.
Apparently, this was also the case for the Cisco IDS-4215 Intrusion Detection Sensor that I was given a few months ago.
I’m not too proud to admit that at first, I didn’t care about the machine itself, but rather about the add-on PCI network card with 4 Fast Ethernet interfaces. The sensor has obviously seen better days, as it had a broken front panel and needed some cleaning, but upon a closer inspection under the hood (which is held closed by the 4 screws on top), this IDS consists of an embedded Celeron PC with two onboard Ethernet cards, a 2.5″ IDE hard disk, a CF card, and 2 PCI expansion slots (more on them later). Oh, and don’t forget the nasty server-grade fan, which pushed very little air for the noise it was making.

###An OpenBSD desktop using WindowMaker

Since I started using N?X, I’ve regularly used WindowMaker. I’ve always liked the look and feel, the dock system and the dockapps. It may look a bit oldish nowadays. And that’s enough to try to change this. So here it is, a 2019 flavored WindowMaker Desktop, running on OpenBSD 6.4/amd64.
This configuration uses the Nord color-scheme, the Adapta-Nokto-Eta GTK theme and the Moblin Unofficial Icons icon set. I did remove applications icons. I just don’t need them on the bottom of the screen as I heavily use “F11” to pop-up the windows list. To be able to do that and keep the dockapps, I tweaked my ~/GNUstep/Defaults/WMWindowAttributes and created a ~/GNUstep/Library/WindowMaker/Themes/Nord.themed/style.
And here it is, the NeXT OpenBSD Desktop!

###RealTime Data Compression

In a previous episode, we’ve seen that it is possible to create opaque types. However, creation and destruction of such type must be delegated to some dedicated functions, which themselves rely on dynamic allocation mechanisms.
Sometimes, it can be convenient to bypass the heap, and all its malloc() / free() shenanigans. Pushing a structure onto the stack, or within thread-local storage, are natural capabilities offered by a normal struct. It can be desirable at times.
The previously described opaque type is so secret that it has no size, hence is not suitable for such scenario.
Fortunately, static opaque types are possible.
The main idea is to create a “shell type”, with a known size and an alignment, able to host the target (private) structure.
For safer maintenance, the shell type and the target structure must be kept in sync, by using typically a static assert. It will ensure that the shell type is always large enough to host the target structure. This check is important to automatically detect future evolution of the target structure.

###For the Love of Pipes

My top used shell command is |. This is called a pipe.
In brief, the | allows for the output of one program (on the left) to become the input of another program (on the right). It is a way of connecting two commands together.
According to doc.cat-v.org/unix/pipes/, the origin of pipes came long before Unix. Pipes can be traced back to this note from Doug McIlroy in 1964

##Beastie Bits

##BUG Calendar

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Your browser does not support the HTML5 video tag.
Thu, 21 Feb 2019 13:00:00 -0800
285: BSD Strategy
Strategic thinking to keep FreeBSD relevant, reflecting on the soul of a new machine, 10GbE Benchmarks On Nine Linux Distros and FreeBSD, NetBSD integrating LLVM sanitizers in base, FreeNAS 11.2 distrowatch review, and more.

##Headlines
###Strategic thinking, or what I think what we need to do to keep FreeBSD relevant

Since I participate in the FreeBSD project there are from time to time some voices which say FreeBSD is dead, Linux is the way to go. Most of the time those voices are trolls, or people which do not really know what FreeBSD has to offer. Sometimes those voices wear blinders, they only see their own little world (were Linux just works fine) and do not see the big picture (like e.g. competition stimulates business, …) or even dare to look what FreeBSD has to offer.
Sometimes those voices raise a valid concern, and it is up to the FreeBSD project to filter out what would be beneficial. Recently there were some mails on the FreeBSD lists in the sense of “What about going into direction X?”. Some people just had the opinion that we should stay where we are. In my opinion this is similarly bad to blindly saying FreeBSD is dead and following the masses. It would mean stagnation. We should not hold people back in exploring new / different directions. Someone wants to write a kernel module in (a subset of) C++ or in Rust… well, go ahead, give it a try, we can put it into the Ports Collection and let people get experience with it.
This discussion on the mailinglists also triggered some kind of “where do we see us in the next years” / strategic thinking reflection. What I present here, is my very own opinion about things we in the FreeBSD project should look at, to stay relevant in the long term. To be able to put that into scope, I need to clarify what “relevant” means in this case.
FreeBSD is currently used by companies like Netflix, NetApp, Cisco, Juniper, and many others as a base for products or services. It is also used by end‐users as a work‐horse (e.g. mailservers, webservers, …). Staying relevant means in this context, to provide something which the user base is interested in to use and which makes it more easy / fast for the user base to deliver whatever they want or need to deliver than with another kind of system. And this in terms of time to market of a solution (time to deliver a service like a web‐/mail‐/whatever‐server or product), and in terms of performance (which not only means speed, but also security and reliability and …) of the solution.
I have categorized the list of items I think are important into (new) code/features, docs, polishing and project infrastructure. Links in the following usually point to documentation/HOWTOs/experiences for/with FreeBSD, and not to the canonical entry points of the projects or technologies. In a few cases the links point to an explanation in the wikipedia or to the website of the topic in question.

###Reflecting on The Soul of a New Machine

Long ago as an undergraduate, I found myself back home on a break from school, bored and with eyes wandering idly across a family bookshelf. At school, I had started to find a calling in computing systems, and now in the den, an old book suddenly caught my eye: Tracy Kidder’s The Soul of a New Machine. Taking it off the shelf, the book grabbed me from its first descriptions of Tom West, captivating me with the epic tale of the development of the Eagle at Data General. I — like so many before and after me — found the book to be life changing: by telling the stories of the people behind the machine, the book showed the creative passion among engineers that might otherwise appear anodyne, inspiring me to chart a course that might one day allow me to make a similar mark.
Since reading it over two decades ago, I have recommended The Soul of a Machine at essentially every opportunity, believing that it is a part of computing’s literary foundation — that it should be considered our Odyssey. Recently, I suggested it as beach reading to Jess Frazelle, and apparently with perfect timing: when I saw the book at the top of her vacation pile, I knew a fuse had been lit. I was delighted (though not at all surprised) to see Jess livetweet her admiration of the book, starting with the compelling prose, the lucid technical explanations and the visceral anecdotes — but then moving on to the deeper technical inspiration she found in the book. And as she reached the book’s crescendo, Jess felt its full power, causing her to reflect on the nature of engineering motivation.
Excited to see the effect of the book on Jess, I experienced a kind of reflected recommendation: I was inspired to (re-)read my own recommendation! Shortly after I started reading, I began to realize that (contrary to what I had been telling myself over the years!) I had not re-read the book in full since that first reading so many years ago. Rather, over the years I had merely revisited those sections that I remembered fondly. On the one hand, these sections are singular: the saga of engineers debugging a nasty I-cache data corruption issue; the young engineer who implements the simulator in an impossibly short amount of time because no one wanted to tell him that he was being impossibly ambitious; the engineer who, frustrated with a nanosecond-scale timing problem in the ALU that he designed, moved to a commune in Vermont, claiming a desire to deal with “no unit of time shorter than a season”. But by limiting myself to these passages, I was succumbing to the selection bias of my much younger self; re-reading the book now from start to finish has given new parts depth and meaning. Aspects that were more abstract to me as an undergraduate — from the organizational rivalries and absurdities of the industry to the complexities of West’s character and the tribulations of the team down the stretch — are now deeply evocative of concrete episodes of my own career.

  • See Article for rest…

##News Roundup

###Out-Of-The-Box 10GbE Network Benchmarks On Nine Linux Distributions Plus FreeBSD 12

Last week I started running some fresh 10GbE Linux networking performance benchmarks across a few different Linux distributions. That testing has now been extended to cover nine Linux distributions plus FreeBSD 12.0 to compare the out-of-the-box networking performance.
Tested this round alongside FreeBSD 12.0 was Antergos 19.1, CentOS 7, Clear Linux, Debian 9.6, Fedora Server 29, openSUSE Leap 15.0, openSUSE Tumbleweed, Ubuntu 18.04.1 LTS, and Ubuntu 18.10.
All of the tests were done with a Tyan S7106 1U server featuring two Intel Xeon Gold 6138 CPUs, 96GB of DDR4 system memory, and Samsung 970 EVO SSD. For the 10GbE connectivity on this server was an add-in HP NC523SFP PCIe adapter providing two 10Gb SPF+ ports using a QLogic 8214 controller.
Originally the plan as well was to include Windows Server 2016/2019. Unfortunately the QLogic driver download site was malfunctioning since Cavium’s acquisition of the company and the other Windows Server 2016 driver options not panning out and there not being a Windows Server 2019 option. So sadly that Windows testing was thwarted so I since started testing over with a Mellanox Connectx-2 10GbE NIC, which is well supported on Windows Server and so that testing is ongoing for the next article of Windows vs. Linux 10 Gigabit network performance plus some “tuned” Linux networking results too.

###Integration of the LLVM sanitizers with the NetBSD base system

Over the past month I’ve merged the LLVM compiler-rt sanitizers (LLVM svn r350590) with the base system. I’ve also managed to get a functional set of Makefile rules to build all of them, namely:
ASan, UBSan, TSan, MSan, libFuzzer, SafeStack, XRay.
In all supported variations and modes that are supported by the original LLVM compiler-rt package.

###Distrowatch FreeNAS 11.2 review

The project’s latest release is FreeNAS 11.2 and, at first, I nearly overlooked the new version because it appeared to be a minor point release. However, a lot of work went into the new version and 11.2 offers a lot of changes when compared next to 11.1, “including a major revamp of the web interface, support for self-encrypting drives, and new, backwards-compatible REST and WebSocket APIs. This update also introduces iocage for improved plugins and jails management and simplified plugin development.”

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 14 Feb 2019 07:00:00 -0800
284: FOSDEM 2019
We recap FOSDEM 2019, FreeBSD Foundation January update, OPNsense 19.1 released, the hardware-assisted virtualization challenge, ZFS and GPL terror, ClonOS 19.01-RELEASE, and more. Headlines FOSDEM 2019 Recap
  • Allan and I were at FOSDEM 2019 in Brussels, Belgium over the weekend.
  • On the Friday before, we held a FreeBSD Devsummit in a hotel conference room, with 25 people attending. We talked about various topics of interest to the project. You can find the notes on the wiki page.
  • Saturday was the first day of FOSDEM. The FreeBSD Project had a table next to the Illumos Project again. A lot of people visited our table, asked questions, or just said “Hi, I watch BSDNow.tv every week”. We handed out a lot of stickers, pens, swag, and flyers. There was also a full day BSD devroom, with a variety of talks that were well attended.
  • In the main conference track, Allan held a talk explaining how the ZFS ARC works. A lot of people attended the talk and had more questions afterwards. Another well attended talk was by Jonathan Looney about Netflix and FreeBSD.
  • Sunday was another day in the same format, but no bsd devroom. A lot of people visited our table, developers and users alike. A lot of meeting and greeting went on.
  • Overall, FOSDEM was a great success with FreeBSD showing a lot of presence. Thanks to all the people who attended and talked to us. Special thanks to the people who helped out at the FreeBSD table and Rodrigo Osorio for running the BSD devroom again.
FreeBSD Foundation Update, January 2019

Dear FreeBSD Community Member,
Happy New Year! It’s always exciting starting the new year with ambitious plans to support FreeBSD in new and existing areas. We achieved our fundraising goal for 2018, so we plan on funding a lot of work this year! Though it’s the new year, this newsletter highlights some of the work we accomplished in December. We also put together a list of technologies and features we are considering supporting, and are looking for feedback on what users want to help inform our 2019 development plans. Our advocacy and education efforts are in full swing as we prepare for upcoming conferences including FOSDEM, SANOG33, and SCaLE.
Finally, we created a year-end video to talk about the work we did in 2018. That in itself was an endeavor, so please take a few minutes to watch it! We’re working on improving the methods we use to inform the community on the work we are doing to support the Project, and are always open to feedback. Now, sit back, grab a refreshing beverage, and enjoy our newsletter!
Happy reading!!
Deb

OPNsense 19.1 released

For more than four years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing.
The 19.1 release, nicknamed “Inspiring Iguana”, consists of a total of 620 individual changes since 18.7 came out 6 months ago, spread out over 12 intermediate releases including the recent release candidates. That is the average of 2 stable releases per month, security updates and important bug fixes included! If we had to pick a few highlights it would be: The firewall alias API is finally in place. The migration to HardenedBSD 11.2 has been completed. 2FA now works with a remote LDAP / local TOTP combination. And the OpenVPN client export was rewritten for full API support as well.

  • These are the most prominent changes since version 18.7:

  • fully functional firewall alias API

  • PIE firewall shaper support

  • firewall NAT rule logging support

  • 2FA via LDAP-TOTP combination

  • WPAD / PAC and parent proxy support in the web proxy

  • P12 certificate export with custom passwords

  • Dpinger is now the default gateway monitor

  • ET Pro Telemetry edition plugin[2]

  • extended IPv6 DUID support

  • Dnsmasq DNSSEC support

  • OpenVPN client export API

  • Realtek NIC driver version 1.95

  • HardenedBSD 11.2, LibreSSL 2.7

  • Unbound 1.8, Suricata 4.1

  • Phalcon 3.4, Perl 5.28

  • firmware health check extended to cover all OS files, HTTPS mirror default

  • updates are browser cache-safe regarding CSS and JavaScript assets

  • collapsible side bar menu in the default theme

  • language updates for Chinese, Czech, French, German, Japanese, Portuguese and Russian

  • API backup export, Bind, Hardware widget, Nginx, Ntopng, VnStat and Dnscrypt-proxy plugins

  • Here are the full changes against version 19.1-RC2:

  • ipsec: add firewall interface as soon as phase 1 is enabled

  • ipsec: phase 1 selection GUI JavaScript compatibility fix

  • monit: widget improvements and bug fix (contributed by Frank Brendel)

  • ui: fix regression in single host or network subnet select in static pages

  • plugins: os-frr 1.7 updates OSFP outbound rules (contributed by Fabian Franz)

  • plugins: os-telegraf 1.7.4 fixes packet filter input

  • plugins: os-theme-rebellion 1.8.2 adds image colour invert

  • plugins: os-vnstat 1.1[3]

  • plugins: os-zabbix-agent now uses Zabbix version 4.0

  • src: revert mmccalculateclock() as HS200/HS400 support breaks legacy support

  • src: update sqlite3-3.20.0 to sqlite3-3.26.0[4]

  • src: import tzdata 2018h, 2018i[5]

  • src: avoid unsynchronized updates to knstatus[6]

  • ports: carootnss 3.42

  • ports: dhcp6c 20190128 prevent rawops double-free (contributed by Team Rebellion)

  • ports: sudo patch to fix listpw=never[7]

News Roundup The hardware-assisted virtualization challenge

Over two years ago, I made a pledge to use NetBSD as my sole OS and only operating system, and to resist booting into any other OS until I had implemented hardware-accelerated virtualization in the NetBSD kernel (the equivalent of Linux’ KVM, or Hyper-V).
Today, I am here to report: Mission Accomplished!
It’s been a long road, but we now have hardware-accelerated virtualization in the kernel! And while I had only initially planned to get Oracle VirtualBox working, I have with the help of the Intel HAXM engine (the same backend used for virtualization in Android Studio) and a qemu frontend, successfully managed to boot a range of mainstream operating systems.

ZFS and GPL terror: How much freedom is there in Linux?
  • ZFS – the undesirable guest

ZFS is todays most advanced filesystem. It originated on the Solaris operating system and thanks to Sun’s decision to open it up, we have it available on quite a number of Unix-like operating systems. That’s just great! Great for everyone.
For everyone? Nope. There are people out there who don’t like ZFS. Which is totally fine, they don’t need to use it after all. But worse: There are people who actively hate ZFS and think that others should not use it. Ok, it’s nothing new that some random guys on the net are acting like assholes, trying to tell you what you must not do, right? Whoever has been online for more than a couple of days probably already got used to it. Unfortunately its still worse: One such spoilsport is Greg Kroah-Hartman, Linux guru and informal second-in-command after Linus Torvalds.
There have been some attempts to defend the stance of this kernel developer. One was to point at the fact that the “ZFS on Linux” (ZoL) port uses two kernel functions, _kernelfpubegin() and _kernelfpuend(), which have been deprecated for a very long time and that it makes sense to finally get rid of them since nothing in-kernel uses it anymore. Nobody is going to argue against that. The problem becomes clear by looking at the bigger picture, though:
The need for functions doing just what the old ones did has of course not vanished. The functions have been replaced with other ones. And those ones are deliberately made GPL-only. Yes, that’s right: There’s no technical reason whatsoever! It’s purely ideology – and it’s a terrible one.

ClonOS 19.01-RELEASE

ClonOS is a turnkey Open Source platform based on FreeBSD and the CBSD framework. ClonOS offers a complete web UI for easily controlling, deploying and managing FreeBSD jails containers and Bhyve/Xen hyperviser virtual environments.
ClonOS is currently the only platform available which allow both Xen and Bhyve hypervisor to coexist on the same host. Being a FreeBSD base platform, ClonOS ability to create and manage jails allows you to run FreeBSD applications without losing performance.

  • Features:

  • easy management via web UI interface

  • live Bhyve migration [coming soon, roadmap]

  • Bhyve management (create, delete VM)

  • Xen management (create, delete VM) [coming soon, roadmap]

  • connection to the “physical” guest console via VNC from the browser or directly

  • Real time system monitoring

  • access to load statistics through SQLite3 and beanstalkd

  • support for ZFS features (cloning, snapshots)

  • import/export of virtual environments

  • public repository with virtual machine templates

  • puppet-based helpers for configuring popular services

  • ClonOS is a free open-source FreeBSD-based platform for virtual environments creation and management. In the core:

  • FreeBSD OS as hoster platform

  • bhyve(8) as hypervisor engine

  • Xen as hypervisor engine

  • vale(4) as Virtual Ethernet Switch

  • jail(8) as container engine

  • CBSD Project as management tools

  • Puppet as configuration management

Beastie Bits Feedback/Questions
  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 07 Feb 2019 08:00:00 -0800
283: Graphical Interface-View
We’re at FOSDEM 2019 this week having fun. We’d never leave you in a lurch, so we have recorded an interview with Niclas Zeising of the FreeBSD graphics team for you. Enjoy.

##Interview - Niclas Zeising - zeising@FreeBSD.org / @niclaszeising
Interview topic: FreeBSD Graphics Stack

  • BR: Welcome Niclas. Since this is your first time on BSDNow, can you tell us a bit about yourself and how you started with Unix/BSD?
  • AJ: What made you start working in the FreeBSD graphics stack?
  • BR: What is the current status with the FreeBSD graphics stack?
  • AJ: What challenges do you face in the FreeBSD graphics stack?
  • BR: How many people are working in the graphics team and what kind of help do you need there?
  • AJ: You’re also involved in FreeBSD ports and held a poudriere tutorial at last years EuroBSDcon. What kind of feedback did you get and will you give that tutorial again?
  • BR: You’ve been organizing the Stockholm BSD user group meeting. Can you tell us a bit about that, what’s involved, how is it structured?
  • AJ: What conferences do you go to where people could talk to you?
  • BR: Is there anything else you’d like to mention before we let you go?

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 31 Jan 2019 08:00:00 -0800
282: Open the Rsync
Project Trident 18.12 released, Spotifyd on NetBSD, OPNsense 18.7.10 is available, Ultra EPYC AMD Powered Sun Ultra 24 Workstation, OpenRsync, LLD porting to NetBSD, and more.

##Headlines

###AsiaBSDCon 2019 Call for Papers

  • You have until Jan 30th to submit
  • Full paper requirement is relaxed a bit this year (this year ONLY!) due to the short submission window. You don’t need all 10-12 pages, but it is still preferred.
  • Send a message to secretary@asiabsdcon.org with your proposal. Could be either for a talk or a tutorial.
  • Two days of tutorials/devsummit and two days of conference during Sakura season in Tokyo, Japan
  • The conference is also looking for sponsors
  • If accepted, flight and hotel is paid for by the conference

###Project Trident 18.12 Released

###Building Spotifyd on NetBSD

These are the steps I went through to build and run Spotifyd (this commit at the time of writing) on NetBSD AMD64. It’s a Spotify Connect client so it means I still need to control Spotify from another device (typically my phone), but the audio is played through my desktop… which is where my speakers and headphones are plugged in - it means I don’t have to unplug stuff and re-plug into my phone, work laptop, etc. This is 100% a “good enough for now solution” for me; I have had a quick play with the Go based microcontroller from spotcontrol and that allows a completely NetBSD only experience (although it is just an example application so doesn’t provide many features - great as a basis to build on though).

##News Roundup

###OPNsense 18.7.10 released

2019 means 19.1 is almost here. In the meantime accept this small
incremental update with goodies such as Suricata 4.1, custom passwords
for P12 certificate export as well as fresh fixes in the FreeBSD base.
A lot of cleanups went into this update to make sure there will be a
smooth transition to 19.1-RC for you early birds. We expect RC1 in 1-2
weeks and the final 19.1 on January 29.

###Introducing the Ultra EPYC AMD Powered Sun Ultra 24 Workstation

A few weeks ago, I got an itch to build a workstation with AMD EPYC. There are a few constraints. First, I needed a higher-clock part. Second, I knew the whole build would be focused more on being an ultra high-end workstation rather than simply utilizing gaming components. With that, I decided it was time to hit on a bit of nostalgia for our readers. Mainly, I wanted to do an homage to Sun Microsystems. Sun made the server gear that the industry ran on for years, and as a fun fact, if you go behind the 1 Hacker Way sign at Facebook’s campus, they left the Sun Microsystems logo. Seeing that made me wonder if we could do an ultimate AMD EPYC build in a Sun Microsystems workstation.

###OpenRsync

This is a clean-room implementation of rsync with a BSD (ISC) license. It is designed to be compatible with a modern rsync (3.1.3 is used for testing). It currently compiles and runs only on OpenBSD.
This project is still very new and very fast-moving.
It’s not ready for wide-spread testing. Or even narrow-spread beyond getting all of the bits to work. It’s not ready for strong attention. Or really any attention but by careful programming.
Many have asked about portability. We’re just not there yet, folks. But don’t worry, the system is easily portable. The hard part for porters is matching OpenBSD’s pledge and unveil.

###The first report on LLD porting

LLD is the link editor (linker) component of Clang toolchain. Its main advantage over GNU ld is much lower memory footprint, and linking speed. It is of specific interest to me since currently 8 GiB of memory are insufficient to link LLVM statically (which is the upstream default).
The first goal of LLD porting is to ensure that LLD can produce working NetBSD executables, and be used to build LLVM itself. Then, it is desirable to look into trying to build additional NetBSD components, and eventually into replacing /usr/bin/ld entirely with lld.
In this report, I would like to shortly summarize the issues I have found so far trying to use LLD on NetBSD.

###Ring in the new

It’s the second week of 2019 already, which means I’m curious what Nate is going to do with his series This week in usability … reset the numbering from week 1? That series is a great read, to keep up with all the little things that change in KDE source each week — aside from the release notes.
For the big ticket items of KDE on FreeBSD, you should read this blog instead.

  • In ports this week (mostly KDE, some unrelated):
  • KDE Plasma has been updated to the latest release, 5.14.5.
  • KDE Applications 18.12.1 were released today, so we’re right on top of them.
  • Marble was fixed for FreeBSD-running-on-Power9.
  • Musescore caught up on 18 months of releases.
  • Phonon updated to 4.10.1, along with its backends.
  • And in development, Qt WebEngine 5.12 has been prepared in the incongruously-named plasma-5.13 branch in Area51; that does contain all the latest bits described above, as well.

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 24 Jan 2019 08:00:00 -0800
281: EPYC Server Battle
SCP client vulnerabilities, BSDs vs Linux benchmarks on a Tyan EPYC Server, fame for the Unix inventors, Die IPv4, GhostBSD 18.12 released, Unix in pictures, and more.

##Headlines
###scp client multiple vulnerabilities

  • Overview
  • SCP clients from multiple vendors are susceptible to a malicious scp server performing
    unauthorized changes to target directory and/or client output manipulation.
  • Description
  • Many scp clients fail to verify if the objects returned by the scp server match those
    it asked for. This issue dates back to 1983 and rcp, on which scp is based. A separate
    flaw in the client allows the target directory attributes to be changed arbitrarily.
    Finally, two vulnerabilities in clients may allow server to spoof the client output.
  • Impact
  • Malicious scp server can write arbitrary files to scp target directory, change the
    target directory permissions and to spoof the client output.
  • Details

The discovered vulnerabilities, described in more detail below, enables the attack
described here in brief.

  • The attacker controlled server or Man-in-the-Middle(*) attack drops .bash_aliases file to victim’s home directory when the victim performs scp operation from the server. The transfer of extra files is hidden by sending ANSI control sequences via stderr. For example:

user@local:~$ scp user@remote:readme.txt .
readme.txt 100% 494 1.6KB/s 00:00
user@local:~$

  • Once the victim launches a new shell, the malicious commands in .bash_aliases get executed.
  • *) Man-in-the-Middle attack does require the victim to accept the wrong host fingerprint.

###FreeBSD 12.0 vs. DragonFlyBSD 5.4 vs. TrueOS 18.12 vs. Linux On A Tyan EPYC Server

Last month when running FreeBSD 12.0 benchmarks on a 2P EPYC server I wasn’t able to run any side-by-side benchmarks with the new DragonFlyBSD 5.4 as this BSD was crashing during the boot process on that board. But fortunately on another AMD EPYC server available, the EPYC 1P TYAN Transport SX TN70A-B8026, DragonFlyBSD 5.4.1 runs fine. So for this first round of BSD benchmarking in 2019 are tests of FreeBSD 11.2, FreeBSD 12.0, DragonFlyBSD 5.4.1, the new TrueOS 18.12, and a few Linux distributions (CentOS 7, Ubuntu 18.04.1 LTS, and Clear Linux) on this EPYC 7601 server in a variety of workloads.

DragonFlyBSD 5.4.1 ran fine on this Tyan server and could boot fine unlike the issue encountered on the Dell PowerEdge R7425 for this particular BSD. But on the Tyan server, DragonFlyBSD 5.2.2 wouldn’t boot so only this latest DragonFlyBSD release series was used as part of the comparison.

  • A summary of the operating systems tested for this EPYC 7601 OS benchmark comparison included:

  • DragonFlyBSD 5.4.1 - The latest release of Matthew Dillon’s operating system while using the HAMMER2 file-system and GCC 8.1 compiler that is now the default system compiler for this BSD.

  • FreeBSD 11.2 - The previous stable release of FreeBSD. Installed with a ZFS file-system.

  • FreeBSD 12.0 - The latest stable release of FreeBSD and installed with its ZFS option.

  • TrueOS 18.12 - The latest release of the iX systems’ FreeBSD derivative. TrueOS 18.12 is based on FreeBSD 13.0-CURRENT and uses ZFS by default and was using the Clang 7.0.1 compiler compared to Clang 6.0.1 on FreeBSD 12.0.

  • CentOS Linux 7 - The latest EL7 operating system performance.

  • Ubuntu 18.04.1 LTS - The latest Ubuntu Long Term Support release.

  • Clear Linux 27120 - The latest rolling release as of testing out of Intel’s Open-Source Technology Center. Clear Linux often reflects as close to the gold standard for performance as possible with its insanely tuned software stack for offering optimal performance on x86_64 performance for generally showing best what the hardware is capable of.

Throughout all of this testing, the Tyan 2U server was kept to its same configuration of an AMD EPYC 7601 (32 cores / 64 threads) at stock speeds, 8 x 16GB DDR4-2666 ECC memory, and 280GB Intel Optane 900p SSD benchmarks.

##News Roundup
National Inventors Hall of Fame honors creators of Unix

Dennis Ritchie (Posthumous) and Ken Thompson: UNIX Operating System
Thompson and Ritchie’s creation of the UNIX operating system and the C programming language were pivotal developments in the progress of computer science. Today, 50 years after its beginnings, UNIX and UNIX-like systems continue to run machinery from supercomputers to smartphones. The UNIX operating system remains the basis of much of the world’s computing infrastructure, and C language – written to simplify the development of UNIX – is one of the most widely used languages today.

###Die IPV4, Die

Imagine, it is 2019. Easy, ha? Imagine, it is 2019 and you want to turn off IPv4. Like, off off. Really off. Not disabling IPv6, but disabling IPv4.

  • Two steps back

You might be coming here wondering, why would anybody want to do what we are asking to be done. Well, it is dead simple: We are running data centers (like Data Center Light) with a lot of IPv6 only equipment. There simply is no need for IPv4. So why would we want to have it enabled?
Also, here at ungleich, we defined 2019 as the year to move away from IPv4.

  • The challenge

Do you like puzzles? Competitions? Challenges? Hacking? Well. If ANY of this is of your interest, here is a real challenge for you:
We offer a 100 CHF (roughly 100 USD) for anyone who can give us a detailed description of how to turn IPv4 completely off in an operating system and allowing it to communicate with IPv6 only. This should obviously include a tiny proof that your operating system is really unable to use IPv4 at all. Just flushing IPv4 addresses and keeping the IPv4 stack loaded, does not count.

###GhostBSD 18.12 released

GhostBSD 18.12 is an updated iso of GhostBSD 18.10 with some little changes to the live DVD/USB and with updated packages.

  • What has changed since 18.10
  • removed default call of kernel modules for AMD and Intel
  • replaced octopkg by software-station
  • added back gop hacks to the live system
  • added ghostbsd-drivers and ghostbsd-utils
  • we updated the packages to the latest build

###And Now for a laugh : #unixinpictures

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 17 Jan 2019 07:00:00 -0800
Episode 280: FOSS Clothing | BSD Now 280
A EULA in FOSS clothing, NetBSD with more LLVM support, Thoughts on FreeBSD 12.0, FreeBSD Performance against Windows and Linux on Xeon, Microsoft shipping NetBSD, and more. Headlines A EULA in FOSS clothing? There was a tremendous amount of reaction to and discussion about my blog entry on the midlife crisis in open source. As part of this discussion on HN, Jay Kreps of Confluent took the time to write a detailed response — which he shortly thereafter elevated into a blog entry. Let me be clear that I hold Jay in high regard, as both a software engineer and an entrepreneur — and I appreciate the time he took to write a thoughtful response. That said, there are aspects of his response that I found troubling enough to closely re-read the Confluent Community License — and that in turn has led me to a deeply disturbing realization about what is potentially going on here. To GitHub: Assuming that this is in fact a EULA, I think it is perilous to allow EULAs to sit in public repositories. It’s one thing to have one click through to accept a license (though again, that itself is dubious), but to say that a git clone is an implicit acceptance of a contract that happens to be sitting somewhere in the repository beggars belief. With efforts like choosealicense.com, GitHub has been a model in guiding projects with respect to licensing; it would be helpful for GitHub’s counsel to weigh in on their view of this new strain of source-available proprietary software and the degree to which it comes into conflict with GitHub’s own terms of service. To foundations concerned with software liberties, including the Apache Foundation, the Linux Foundation, the Free Software Foundation, the Electronic Frontier Foundation, the Open Source Initiative, and the Software Freedom Conservancy: the open source community needs your legal review on this! I don’t think I’m being too alarmist when I say that this is potentially a dangerous new precedent being set; it would be very helpful to have your lawyers offer their perspectives on this, even if they disagree with one another. We seem to be in some terrible new era of frankenlicenses, where the worst of proprietary licenses are bolted on to the goodwill created by open source licenses; we need your legal voices before these creatures destroy the village! NetBSD and LLVM NetBSD entering 2019 with more complete LLVM support I’m recently helping the NetBSD developers to improve the support for this operating system in various LLVM components. As you can read in my previous report, I’ve been focusing on fixing build and test failures for the purpose of improving the buildbot coverage. Previously, I’ve resolved test failures in LLVM, Clang, LLD, libunwind, openmp and partially libc++. During the remainder of the month, I’ve been working on the remaining libc++ test failures, improving the NetBSD clang driver and helping Kamil Rytarowski with compiler-rt. The process of upstreaming support to LLVM sanitizers has been finalized I’ve finished the process of upstreaming patches to LLVM sanitizers (almost 2000LOC of local code) and submitted to upstream new improvements for the NetBSD support. Today out of the box (in unpatched version) we have support for a variety of compiler-rt LLVM features: ASan (finds unauthorized memory access), UBSan (finds unspecified code semantics), TSan (finds threading bugs), MSan (finds uninitialized memory use), SafeStack (double stack hardening), Profile (code coverage), XRay (dynamic code tracing); while other ones such as Scudo (hardened allocator) or DFSan (generic data flow sanitizer) are not far away from completeness. The NetBSD support is no longer visibly lacking behind Linux in sanitizers, although there are still failing tests on NetBSD that are not observed on Linux. On the other hand there are features working on NetBSD that are not functional on Linux, like sanitizing programs during early initialization process of OS (this is caused by /proc dependency on Linux that is mounted by startup programs, while NetBSD relies on sysctl(3) interfaces that is always available). News Roundup Thoughts on FreeBSD 12.0 Playing with FreeBSD with past week I don’t feel as though there were any big surprises or changes in this release compared to FreeBSD 11. In typical FreeBSD fashion, progress tends to be evolutionary rather than revolutionary, and this release feels like a polished and improved incremental step forward. I like that the installer handles both UFS and ZFS guided partitioning now and in a friendly manner. In the past I had trouble getting FreeBSD’s boot menu to work with boot environments, but that has been fixed for this release. I like the security options in the installer too. These are not new, but I think worth mentioning. FreeBSD, unlike most Linux distributions, offers several low-level security options (like hiding other users’ processes and randomizing PIDs) and I like having these presented at install time. It’s harder for people to attack what they cannot see, or predict, and FreeBSD optionally makes these little adjustment for us. Something which stands out about FreeBSD, compared to most Linux distributions I run, is that FreeBSD rarely holds the user’s hand, but also rarely surprises the user. This means there is more reading to do up front and new users may struggle to get used to editing configuration files in a text editor. But FreeBSD rarely does anything unless told to do it. Updates rarely change the system’s behaviour, working technology rarely gets swapped out for something new, the system and its applications never crashed during my trial. Everything was rock solid. The operating system may seem like a minimal, blank slate to new users, but it’s wonderfully dependable and predictable in my experience. I probably wouldn’t recommend FreeBSD for desktop use. It’s close relative, GhostBSD, ships with a friendly desktop and does special work to make end user applications run smoothly. But for people who want to run servers, possible for years without change or issues, FreeBSD is a great option. It’s also an attractive choice, in my opinion, for people who like to build their system from the ground up, like you would with Debian’s server install or Arch Linux. Apart from the base tools and documentation, there is nothing on a FreeBSD system apart from what we put on it. FreeBSD 12.0 Performance Against Windows & Linux On An Intel Xeon Server Last week I posted benchmarks of Windows Server 2019 against various Linux distributions using a Tyan dual socket Intel Xeon server. In this article are some complementary results when adding in the performance of FreeBSD 11.2 against the new FreeBSD 12.0 stable release for this leading BSD operating system. As some fun benchmarks to end out 2018, here are the results of FreeBSD 11.2/12.0 (including an additional run when using GCC rather than Clang) up against Windows Server and several enterprise-ready Linux distributions. While FreeBSD 12.0 had picked up just one win of the Windows/Linux comparisons run, the FreeBSD performance is moving in the right direction. FreeBSD 12.0 was certainly faster than FreeBSD 11.2 on this dual Intel Xeon Scalable server based on a Tyan 1U platform. Meanwhile, to no surprise given the data last week, Clear Linux was by far the fastest out-of-the-box operating system tested. I did run some extra benchmarks on FreeBSD 11.2/12.0 with this hardware: in total I ran 120 benchmarks for these BSD tests. Of the 120 tests, there were just 15 cases where FreeBSD 11.2 was faster than 12.0. Seeing FreeBSD 12.0 faster than 11.2 nearly 90% of the time is an accomplishment and usually with other operating systems we see more of a mixed bag on new releases with not such solidly better performance. It was also great seeing the competitive performance out of FreeBSD when using the Clang compiler for the source-based tests compared to the GCC8 performance. Additional data available via this OpenBenchmarking.org result file. How NetBSD came to be shipped by Microsoft Google cache in case the site is down In 2000, Joe Britt, Matt Hershenson and Andy Rubin formed Danger Incorporated. Danger developed the world’s first recognizable smartphone, the Danger HipTop. T-Mobile sold the first HipTop under the brand name Sidekick in October of 2002. Danger had a well developed kernel that had been designed and built in house. The kernel came to be viewed as not a core intellectual property and Danger started a search for a replacement. For business reasons, mostly to do with legal concerns over the Gnu Public License, Danger rejected Linux and began to consider BSD Unix as a replacement for the kernel. In 2006 I was hired by Mike Chen, the manager of the kernel development group to investigate the feasibility of replacing the Danger kernel with a BSD kernel, to select the version of BSD to use, to develop a prototype and to develop the plan for adapting BSD to Danger’s requirements. NetBSD was easily the best choice among the BSD variations at the time because it had well developed cross development tools. It was easy to use a NetBSD desktop running an Intel release to cross compile a NetBSD kernel and runtime for a device running an ARM processor. (Those interested in mailing list archaeology might be amused to investigate NetBSD technical mailing list for mail from picovex, particularly from Bucky Katz at picovex.) We began product development on the specific prototype of the phone that would become the Sidekick LX2009 in 2007 and contracts for the phone were written with T-Mobile. We were about half way through the two year development cycle when Microsoft purchased Danger in 2008. Microsoft would have preferred to ship the Sidekick running Windows/CE rather than NetBSD, but a schedule analysis performed by me, and another by an independent outside contractor, indicated that doing so would result in unacceptable delay. Beastie Bits Feedback/Questions
  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 10 Jan 2019 00:00:00 -0800
Episode 279: Future of ZFS | BSD Now 279
The future of ZFS in FreeBSD, we pick highlights from the FreeBSD quarterly status report, flying with the raven, modern KDE on FreeBSD, many ways to launch FreeBSD in EC2, GOG installers on NetBSD, and more. Headlines The future of ZFS in FreeBSD The sources for FreeBSD’s ZFS support are currently taken directly from Illumos with local ifdefs to support the peculiarities of FreeBSD where the Solaris Portability Layer (SPL) shims fall short. FreeBSD has regularly pulled changes from Illumos and tried to push back any bug fixes and new features done in the context of FreeBSD. In the past few years the vast majority of new development in ZFS has taken place in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced that they will be moving to ZoL: https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means that there will be little to no net new development of Illumos. While working through the git history of ZoL I have also discovered that many races and locking bugs have been fixed in ZoL and never made it back to Illumos and thus FreeBSD. This state of affairs has led to a general agreement among the stakeholders that I have spoken to that it makes sense to rebase FreeBSD’s ZFS on ZoL. Brian Behlendorf has graciously encouraged me to add FreeBSD support directly to ZoL https://github.com/zfsonfreebsd/ZoF so that we might all have a single shared code base. A port for ZoF can be found at https://github.com/miwi-fbsd/zof-port Before it can be committed some additional functionality needs to be added to the FreeBSD opencrypto framework. These can be found at https://reviews.freebsd.org/D18520 This port will provide FreeBSD users with multi modifier protection, project quotas, encrypted datasets, allocation classes, vectorized raidz, vectorized checksums, and various command line improvements. FreeBSD Quarterly Status Update With FreeBSD having gone all the way to 12, it is perhaps useful to take a look back at all the things that have been accomplished, in terms of many visible changes, as well as all the things that happen behind the scenes to ensure that FreeBSD continues to offer an alternative in both design, implementation, and execution. The things you can look forward to reading about are too numerous to summarize, but cover just about everything from finalizing releases, administrative work, optimizations and depessimizations, features added and fixed, and many areas of improvement that might just surprise you a little. Please have a cup of coffee, tea, hot cocoa, or other beverage of choice, and enjoy this culmulative set of reports covering everything that’s been done since October, 2017. —Daniel Ebdrup News Roundup One year of flying with the Raven: Ready for the Desktop? It has been a little over one year now that I’m with the Ravenports project. Time to reflect my involvement, my expectations and hopes.
  • Ravenports
Ravenports is a universal packaging framework for *nix operating systems. For the user it provides easy access to binary packages of common software for multiple platforms. It has been the long-lasting champion on Repology’s top 10 repositories regarding package freshness (rarely dropping below 96 percent while all other projects keep below 90!). For the porter it offers a well-designed and elegant means of writing cross-platform buildsheets that allow building the same version of the software with (completely or mostly) the same compile-time configuration on different operating systems or distributions. And for the developer it means a real-world project that’s written in modern Ada (ravenadm) and C (pkg) – as well as some Perl for support scripts and make. Things feel very optimized and fast. Not being a programmer though, I cannot really say anything about the actual code and thus leave it to the interested reader’s judgement. Modern KDE on FreeBSD New stuff in the official FreeBSD repositories! The X11 team has landed a newer version of libinput, opening up the way for KDE Plasma 5.14 in ports. That’s a pretty big update and it may frighten people with a new wallpaper. What this means is that the graphical stack is once again on-par with what Plasma upstream expects, and we can get back to chasing releases as soon as they happen, rather than gnashing our teeth at missing dependencies. The KDE-FreeBSD CI servers are in the process of being upgraded to 12-STABLE, and we’re integrating with the new experimental CI systems as well. This means we are chasing sensibly-modern systems (13-CURRENT is out of scope). The many ways to launch FreeBSD in EC2 Talking to FreeBSD users recently, I became aware that while I’ve created a lot of tools, I haven’t done a very good job of explaining how, and more importantly when to use them. So for all of the EC2-curious FreeBSD users out there: Here are the many ways to launch and configure FreeBSD in EC2 — ranging from the simplest to the most complicated (but most powerful):
  • Launch FreeBSD and SSH in
  • Launch FreeBSD and provide user-data
  • Use the AMI Builder to create a customized FreeBSD AMI
  • Build a FreeBSD AMI from a modified FreeBSD source tree
  • Build your own disk image
I hope I’ve provided tools which help you to run FreeBSD in EC2, no matter how common or unusual your needs are. If you find my work useful, please consider supporting my work in this area; while this is both something I enjoy working on and something which is useful for my day job (Tarsnap, my online backup service), having support would make it easier for me to prioritize FreeBSD/EC2 issues over other projects. Using the GOG.com installers for Linux, on NetBSD GOG.com prefers that you use their GOG Galaxy desktop app to download, install and manage all of your GOG games. But customers always have the option to install the game on their own terms, with a platform-specific installer. GOG offers these installers for Mac, Windows and/or Linux, depending on which platforms the game is available for.
  • The installers truly are platform-specific:
  • macOS games are distributed in a standard .pkg
  • Windows games are distributed in a setup wizard .exe
  • Linux games are distributed in a goofy shell archive
Of course, none of those are NetBSD. So, if I wanted to even attempt to play a game distributed by GOG.com on NetBSD, which one should I pick? The obvious choice is the Linux installer, since Linux is the most similar to NetBSD, right? Au contraire! In practice, I found that it is easier to download the Windows installer. Here’s what I mean. For example, I ported the open source version of Aquaria to pkgsrc, but that package is only the game’s engine, not the multimedia data. The multimedia data is still copyrighted. Therefore, you need to get it from somewhere else. GOG is usually a good choice, because they distribute their games without DRM. And as mentioned earlier, picking the Linux installer seemed like a natural choice. Now, actually PLAYING the games on NetBSD is a separate matter entirely. The game I’ve got here, though, my current obsession Pyre, is built with MonoGame and therefore could theoretically work on NetBSD, too, with the help of a library called FNA and a script for OpenBSD called fnaify. I do hope to create a pkgsrc package for FNA and port the fnaify script to NetBSD at some point. Beastie Bits Feedback/Questions
  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 03 Jan 2019 08:00:00 -0800
Episode 278: The Real McCoy | BSD Now 278
We sat down at BSDCan 2018 to interview Kirk McKusick about various topics ranging about the early years of Berkeley Unix, his continuing work on UFS, the governance of FreeBSD, and more.

##Interview - Kirk McKusick - mckusick@mckusick.com
25 years of FreeBSD

  • How Kirk got started in BSD, at the very beginning
  • Predicting the Future
  • How the code and community grew
  • The leadership of the project, and how it changed over time
  • UFS over the years (reading disks from 1982 in 2018)
  • Conferences
  • The rise and fall of Linux
  • The resurgence of FreeBSD

We want to extend a big thank you to the entire BSD community for making this show possible, and to all of our viewers for watching and providing the feedback that makes this show successful. We wish you all a happy and prosperous new year, and we’ll see you next week.

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 27 Dec 2018 01:00:00 -0800
Episode 277: Nmap Level Up | BSD Now 277
The Open Source midlife crisis, Donald Knuth The Yoda of Silicon Valley, Certbot For OpenBSD's httpd, how to upgrade FreeBSD from 11 to 12, level up your nmap game, NetBSD desktop, and more.

##Headlines
Open Source Confronts its midlife crisis

Midlife is tough: the idealism of youth has faded, as has inevitably some of its fitness and vigor. At the same time, the responsibilities of adulthood have grown. Making things more challenging, while you are navigating the turbulence of teenagers, your own parents are likely entering life’s twilight, needing help in new ways from their adult children. By midlife, in addition to the singular joys of life, you have also likely experienced its terrible sorrows: death, heartbreak, betrayal. Taken together, the fading of youth, the growth in responsibility and the endurance of misfortune can lead to cynicism or (worse) drastic and poorly thought-out choices. Add in a little fear of mortality and some existential dread, and you have the stuff of which midlife crises are made…
I raise this not because of my own adventures at midlife, but because it is clear to me that open source — now several decades old and fully adult — is going through its own midlife crisis. This has long been in the making: for years, I (and others) have been critical of service providers’ parasitic relationship with open source, as cloud service providers turn open source software into a service offering without giving back to the communities upon which they implicitly depend. At the same time, open source has been (rightfully) entirely unsympathetic to the proprietary software models that have been burned to the ground — but also seemingly oblivious as to the larger economic waves that have buoyed them.
So it seemed like only a matter of time before the companies built around open source software would have to confront their own crisis of confidence: open source business models are really tough, selling software-as-a-service is one of the most natural of them, the cloud service providers are really good at it — and their commercial appetites seem boundless. And, like a new cherry red two-seater sports car next to a minivan in a suburban driveway, some open source companies are dealing with this crisis exceptionally poorly: they are trying to restrict the way that their open source software can be used. These companies want it both ways: they want the advantages of open source — the community, the positivity, the energy, the adoption, the downloads — but they also want to enjoy the fruits of proprietary software companies in software lock-in and its monopolistic rents. If this were entirely transparent (that is, if some bits were merely being made explicitly proprietary), it would be fine: we could accept these companies as essentially proprietary software companies, albeit with an open source loss-leader. But instead, these companies are trying to license their way into this self-contradictory world: continuing to claim to be entirely open source, but perverting the license under which portions of that source are available. Most gallingly, they are doing this by hijacking open source nomenclature. Of these, the laughably named commons clause is the worst offender (it is plainly designed to be confused with the purely virtuous creative commons), but others (including CockroachDB’s Community License, MongoDB’s Server Side Public License, and Confluent’s Community License) are little better. And in particular, as it apparently needs to be said: no, “community” is not the opposite of “open source” — please stop sullying its good name by attaching it to licenses that are deliberately not open source! But even if they were more aptly named (e.g. “the restricted clause” or “the controlled use license” or — perhaps most honest of all — “the please-don’t-put-me-out-of-business-during-the-next-reInvent-keynote clause”), these licenses suffer from a serious problem: they are almost certainly asserting rights that the copyright holder doesn’t in fact have.
If I sell you a book that I wrote, I can restrict your right to read it aloud for an audience, or sell a translation, or write a sequel; these restrictions are rights afforded the copyright holder. I cannot, however, tell you that you can’t put the book on the same bookshelf as that of my rival, or that you can’t read the book while flying a particular airline I dislike, or that you aren’t allowed to read the book and also work for a company that competes with mine. (Lest you think that last example absurd, that’s almost verbatim the language in the new Confluent Community (sic) License.) I personally think that none of these licenses would withstand a court challenge, but I also don’t think it will come to that: because the vendors behind these licenses will surely fear that they wouldn’t survive litigation, they will deliberately avoid inviting such challenges. In some ways, this netherworld is even worse, as the license becomes a vessel for unverifiable fear of arbitrary liability.
let me put this to you as directly as possible: cloud services providers are emphatically not going to license your proprietary software. I mean, you knew that, right? The whole premise with your proprietary license is that you are finding that there is no way to compete with the operational dominance of the cloud services providers; did you really believe that those same dominant cloud services providers can’t simply reimplement your LDAP integration or whatever? The cloud services providers are currently reproprietarizing all of computing — they are making their own CPUs for crying out loud! — reimplementing the bits of your software that they need in the name of the service that their customers want (and will pay for!) won’t even move the needle in terms of their effort.
Worse than all of this (and the reason why this madness needs to stop): licenses that are vague with respect to permitted use are corporate toxin. Any company that has been through an acquisition can speak of the peril of the due diligence license audit: the acquiring entity is almost always deep pocketed and (not unrelatedly) risk averse; the last thing that any company wants is for a deal to go sideways because of concern over unbounded liability to some third-party knuckle-head. So companies that engage in license tomfoolery are doing worse than merely not solving their own problem: they are potentially poisoning the wellspring of their own community.
in the end, open source will survive its midlife questioning just as people in midlife get through theirs: by returning to its core values and by finding rejuvenation in its communities. Indeed, we can all find solace in the fact that while life is finite, our values and our communities survive us — and that our engagement with them is our most important legacy.

  • See the article for the rest

###Donald Knuth - The Yoda of Silicon Valley

For half a century, the Stanford computer scientist Donald Knuth, who bears a slight resemblance to Yoda — albeit standing 6-foot-4 and wearing glasses — has reigned as the spirit-guide of the algorithmic realm.
He is the author of “The Art of Computer Programming,” a continuing four-volume opus that is his life’s work. The first volume debuted in 1968, and the collected volumes (sold as a boxed set for about $250) were included by American Scientist in 2013 on its list of books that shaped the last century of science — alongside a special edition of “The Autobiography of Charles Darwin,” Tom Wolfe’s “The Right Stuff,” Rachel Carson’s “Silent Spring” and monographs by Albert Einstein, John von Neumann and Richard Feynman.
With more than one million copies in print, “The Art of Computer Programming” is the Bible of its field. “Like an actual bible, it is long and comprehensive; no other book is as comprehensive,” said Peter Norvig, a director of research at Google. After 652 pages, volume one closes with a blurb on the back cover from Bill Gates: “You should definitely send me a résumé if you can read the whole thing.”
The volume opens with an excerpt from “McCall’s Cookbook”:

Here is your book, the one your thousands of letters have asked us to publish. It has taken us years to do, checking and rechecking countless recipes to bring you only the best, only the interesting, only the perfect.

Inside are algorithms, the recipes that feed the digital age — although, as Dr. Knuth likes to point out, algorithms can also be found on Babylonian tablets from 3,800 years ago. He is an esteemed algorithmist; his name is attached to some of the field’s most important specimens, such as the Knuth-Morris-Pratt string-searching algorithm. Devised in 1970, it finds all occurrences of a given word or pattern of letters in a text — for instance, when you hit Command+F to search for a keyword in a document.
Now 80, Dr. Knuth usually dresses like the youthful geek he was when he embarked on this odyssey: long-sleeved T-shirt under a short-sleeved T-shirt, with jeans, at least at this time of year. In those early days, he worked close to the machine, writing “in the raw,” tinkering with the zeros and ones.

  • See the article for the rest

##News Roundup
Let’s Encrypt: Certbot For OpenBSD’s httpd

  • Intro

Let’s Encrypt is “a free, automated, and open Certificate Authority”.
Certbot is “an easy-to-use automatic client that fetches and deploys SSL/TLS certificates for your web server”, well known as “the official Let’s Encrypt client”.
I remember well how excited I felt when I read Let’s Encrypt’s “Our First Certificate Is Now Live” in 2015.
How wonderful the goal of them is; it’s to “give people the digital certificates they need in order to enable HTTPS (SSL/TLS) for websites, for free” “to create a more secure and privacy-respecting Web”!
Since this year, they have begun to support even ACME v2 and Wildcard Certificate!
Well, in OpenBSD as well as other operating systems, it’s easy and comfortable to have their big help 😊

  • Environment
  • OS: OpenBSD 6.4 amd64
  • Web Server: OpenBSD’s httpd
  • Certification: Let’s Encrypt with Certbot 0.27
  • Reference: OpenBSD’s httpd

###FreeBSD 12 released: Here is how to upgrade FreeBSD 11 to 12

The FreeBSD project announces the availability of FreeBSD 12.0-RELEASE. It is the first release of the stable/12 branch. The new version comes with updated software and features for a wild variety of architectures. The latest release provides performance improvements and better support for FreeBSD jails and more. One can benefit greatly using an upgraded version of FreeBSD.

FreeBSD 12.0 supports amd64, i386, powerpc, powerpc64, powerpcspe, sparc64, armv6, armv7, and aarch64 architectures. One can run it on a standalone server or desktop system. Another option is to run it on Raspberry PI computer. FreeBSD 12 also runs on popular cloud service providers such as AWS EC2/Lightsail or Google compute VM.

  • New features and highlights:

  • OpenSSL version 1.1.1a (LTS)

  • OpenSSH server 7.8p1

  • Unbound server 1.8.1

  • Clang and co 6.0.1

  • The FreeBSD installer supports EFI+GELI as an installation option

  • VIMAGE FreeBSD kernel configuration option has been enabled by default. VIMAGE was the main reason I custom compiled FreeBSD for the last few years. No more custom compile for me.

  • Graphics drivers for modern ATI/AMD and Intel graphics cards are now available in the FreeBSD ports collection

  • ZFS has been updated to include new sysctl(s), vfs.zfs.arcminprefetchms and vfs.zfs.arcminprescientprefetchms, which improve performance of the zpool scrub subcommand

  • The pf packet filter is now usable within a jail using vnet

  • KDE updated to version 5.12.5

  • The NFS version 4.1 includes pNFS server support

  • Perl 5.26.2

  • The default PAGER now defaults to less for most commands

  • The dd utility has been updated to add the status=progress option to match GNU/Linux dd command to show progress bar while running dd

  • FreeBSD now supports ext4 for read/write operation

  • Python 2.7

  • much more

###Six Ways to Level Up Your nmap Game

nmap is a network exploration tool and security / port scanner.
If you’ve heard of it, and you’re like me, you’ve most likely used it like this:
ie, you’ve pointed it at an IP address and observed the output which tells you the open ports on a host.
I used nmap like this for years, but only recently grokked the manual to see what else it could do. Here’s a quick look and some of the more useful things I found out.

  • Scan a Network
  • Scan All Ports
  • Get service versions
  • Use -A for more data
  • Find out what nmap is up to
  • Script your own scans with NSE

###[NetBSD Desktop]

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Mon, 24 Dec 2018 08:00:00 -0800
Episode 276: Ho, Ho, Ho - 12.0 | BSD Now 276
FreeBSD 12.0 is finally here, partly-cloudy IPsec VPN, KLEAK with NetBSD, How to create synth repos, GhostBSD author interview, and more.

##Headlines
FreeBSD 12.0 is available

  • After a long release cycle, the wait is over: FreeBSD 12.0 is now officially available.
  • We’ve picked a few interesting things to cover in the show, make sure to read the full Release Notes

Userland:
Group permissions on /dev/acpi have been changed to allow users in the operator GID to invoke acpiconf(8) to suspend the system.
The default devfs.rules(5) configuration has been updated to allow mount_fusefs(8) with jail(8).
The default PAGER now defaults to less(1) for most commands.
The newsyslog(8) utility has been updated to reject configuration entries that specify setuid(2) or executable log files.
The WITH_REPRODUCIBLE_BUILD src.conf(5) knob has been enabled by default.
A new src.conf(5) knob, WITH_RETPOLINE, has been added to enable the retpoline mitigation for userland builds.
Userland applications:
The dtrace(1) utility has been updated to support if and else statements.
The legacy gdb(1) utility included in the base system is now installed to /usr/libexec for use with crashinfo(8). The gdbserver and gdbtui utilities are no longer installed. For interactive debugging, lldb(1) or a modern version of gdb(1) from devel/gdb should be used. A new src.conf(5) knob, WITHOUT_GDB_LIBEXEC has been added to disable building gdb(1). The gdb(1) utility is still installed in /usr/bin on sparc64.
The setfacl(1) utility has been updated to include a new flag, -R, used to operate recursively on directories.
The geli(8) utility has been updated to provide support for initializing multiple providers at once when they use the same passphrase and/or key.
The dd(1) utility has been updated to add the status=progress option, which prints the status of its operation on a single line once per second, similar to GNU dd(1).
The date(1) utility has been updated to include a new flag, -I, which prints its output in ISO 8601 formatting.
The bectl(8) utility has been added, providing an administrative interface for managing ZFS boot environments, similar to sysutils/beadm.
The bhyve(8) utility has been updated to add a new subcommand to the -l and -s flags, help, which when used, prints a list of supported LPC and PCI devices, respectively.
The tftp(1) utility has been updated to change the default transfer mode from ASCII to binary.
The chown(8) utility has been updated to prevent overflow of UID or GID arguments where the argument exceeded UID_MAX or GID_MAX, respectively.
Kernel:
The ACPI subsystem has been updated to implement Device object types for ACPI 6.0 support, required for some Dell, Inc. Poweredge™ AMD® Epyc™ systems.
The amdsmn(4) and amdtemp(4) drivers have been updated to attach to AMD® Ryzen 2™ host bridges.
The amdtemp(4) driver has been updated to fix temperature reporting for AMD® 2990WX CPUs.
Kernel Configuration:
The VIMAGE kernel configuration option has been enabled by default.
The dumpon(8) utility has been updated to add support for compressed kernel crash dumps when the kernel configuration file includes the GZIO option. See rc.conf(5) and dumpon(8) for additional information.
The NUMA option has been enabled by default in the amd64 GENERIC and MINIMAL kernel configurations.
Device Drivers:
The random(4) driver has been updated to remove the Yarrow algorithm. The Fortuna algorithm remains the default, and now only, available algorithm.
The vt(4) driver has been updated with performance improvements, drawing text at rates ranging from 2- to 6-times faster.
Deprecated Drivers:
The lmc(4) driver has been removed.
The ixgb(4) driver has been removed.
The nxge(4) driver has been removed.
The vxge(4) driver has been removed.
The jedec_ts(4) driver has been removed in 12.0-RELEASE, and its functionality replaced by jedec_dimm(4).
The DRM driver for modern graphics chipsets has been marked deprecated and marked for removal in FreeBSD 13. The DRM kernel modules are available from graphics/drm-stable-kmod or graphics/drm-legacy-kmod in the Ports Collection as well as via pkg(8). Additionally, the kernel modules have been added to the lua loader.conf(5) module_blacklist, as installation from the Ports Collection or pkg(8) is strongly recommended.
The following drivers have been deprecated in FreeBSD 12.0, and not present in FreeBSD 13.0: ae(4), de(4), ed(4), ep(4), ex(4), fe(4), pcn(4), sf(4), sn(4), tl(4), tx(4), txp(4), vx(4), wb(4), xe(4)
Storage:
The UFS/FFS filesystem has been updated to support check hashes to cylinder-group maps. Support for check hashes is available only for UFS2.
The UFS/FFS filesystem has been updated to consolidate TRIM/BIO_DELETE commands, reducing read/write requests due to fewer TRIM messages being sent simultaneously.
TRIM consolidation support has been enabled by default in the UFS/FFS filesystem. TRIM consolidation can be disabled by setting the vfs.ffs.dotrimcons sysctl(8) to 0, or adding vfs.ffs.dotrimcons=0 to sysctl.conf(5).
NFS:
The NFS version 4.1 server has been updated to include pNFS server support.
ZFS:
ZFS has been updated to include new sysctl(8)s, vfs.zfs.arc_min_prefetch_ms and vfs.zfs.arc_min_prescient_prefetch_ms, which improve performance of the zpool(8) scrub subcommand.
The new spacemap_v2 zpool feature has been added. This provides more efficient encoding of spacemaps, especially for full vdev spacemaps.
The large_dnode zpool feature been imported, allowing better compatibility with pools created under ZFS-on-Linux 0.7.x
Many bug fixes have been applied to the device removal feature. This feature allows you to remove a non-redundant or mirror vdev from a pool by relocating its data to other vdevs.
Includes the fix for PR 229614 that could cause processes to hang in zil_commit()
Boot Loader Changes:
The lua loader(8) has been updated to detect a list of installed kernels to boot.
The loader(8) has been updated to support geli(8) for all architectures and all disk-like devices.
The loader(8) has been updated to add support for loading Intel® microcode updates early during the boot process.

Networking:
The pf(4) packet filter is now usable within a jail(8) using vnet(9).
The pf(4) packet filter has been updated to use rmlock(9) instead of rwlock(9), resulting in significant performance improvements.
The SO_REUSEPORT_LB option has been added to the network stack, allowing multiple programs or threads to bind to the same port, and incoming connections load balanced using a hash function.

  • Again, read the release notes for a full list, check out the errata notices. A big THANKS to the entire release engineering team and all developers involved in the release, much appreciated!

###Abandon Linux. Move to FreeBSD or Illumos

If you use GNU/Linux and you are only on opensource, you may be doing it wrong. Here’s why.
Is your company based on opensource based software only? Do you have a bunch of developers hitting some kind of server you have installed for them to “do their thing”? Being it for economical reasons (remember to donate), being it for philosophycal ones, you may have skipped good alternatives. The BSD’s and Illumos.
I bet you are running some sort of Debian, openSuSE or CentOS. It’s very discouraging having entered into the IT field recently and discover many of the people you meet do not even recognise the name BSD. Naming Solaris seems like naming the evil itself. The problem being many do not know why. They can’t point anything specific other than it’s fading out. This has recently shown strong when Oracle officials have stated development for new features has ceased and almost 90 % of developers for Solaris have been layed off. AIX seems alien to almost everybody unless you have a white beard. And all this is silly.
And here’s why. You are certainly missing two important features that FreeBSD and Illumos derivatives are enjoying. A full virtualization technology, much better and fully developed compared to the LXC containers in the Linux world, such as Jails on BSD, Zones in Solaris/Illumos, and the great ZFS file system which both share.
You have probably heard of a new Linux filesystem named Btrfs, which by the way, development has been dropped from the Red Hat side. Trying to emulate ZFS, Oracle started developing Btrfs file system before they acquired Sun (the original developer of ZFS), and SuSE joined the effort as well as Red Hat. It is not as well developed as ZFS and it hasn’t been tested in production environments as extensively as the former has. That leaves some uncertainty on using it or not. Red Hat leaving it aside does add some more. Although some organizations have used it with various grades of success.
But why is this anyhow interesting for a sysadmin or any organization? Well… FreeBSD (descendant of Berkeley UNIX) and SmartOS (based on Illumos) aglutinate some features that make administration easier, safer, faster and more reliable. The dream of any systems administrator.
To start, the ZFS filesystem combines the typical filesystem with a volume manager. It includes protection against corruption, snapshots and copy-on-write clones, as well as volume manager.
Jails is another interesting piece of technology. Linux folks usually associate this as a sort of chroot. It isn’t. It is somehow inspired by it but as you may know you can escape from a chroot environment with a blink of an eye. Jails are not called jails casually. The name has a purpose. Contain processes and programs within a defined and totally controlled environment. Jails appeared first in FreeBSD in the year 2000. Solaris Zones debuted on 2005 (now called containers) are the now proprietary version of those.
There are some other technologies on Linux such as Btrfs or Docker. But they have some caveats. Btrfs hasn’t been fully developed yet and it’s hasn’t been proved as much in production environments as ZFS has. And some problems have arisen recently although the developers are pushing the envelope. At some time they will match ZFS capabilities for sure. Docker is growing exponentially and it’s one of the cool technologies of modern times. The caveat is, as before, the development of this technology hasn’t been fully developed. Unlike other virtualization technologies this is not a kernel playing on top of another kernel. This is virtualization at the OS level, meaning differentiated environments can coexist on a single host, “hitting” the same unique kernel which controls and shares the resources. The problem comes when you put Docker on top of any other virtualization technology such as KVM or Xen. It breaks the purpose of it and has a performance penalty.
I have arrived into the IT field with very little knowledge, that is true. But what I see strikes me. Working in a bank has allowed me to see a big production environment that needs the highest of the availability and reliability. This is, sometimes, achieved by bruteforce. And it’s legitime and adequate. Redundancy has a reason and a purpose for example. But some other times it looks, it feels, like killing flies with cannons. More hardware, more virtual machines, more people, more of this, more of that. They can afford it, so they try to maintain the cost low but at the end of the day there is a chunky budget to back operations.
But here comes reality. You’re not a bank and you need to squeeze your investment as much as possible. By using FreeBSD jails you can avoid the performance penalty of KVM or Xen virtualization. Do you use VMWare or Hyper-V? You can avoid both and gain in performance. Not only that, control and manageability are equal as before, and sometimes easier to administer. There are four ways to operate them which can be divided in two categories. Hardcore and Human Being. For the Hardcore use the FreeBSD handbook and investigate as much as you can. For the Human Being way there are three options to use. Ezjail, Iocage and CBSD which are frameworks or programs as you may call to manage jails. I personally use Iocage but I have also used Ezjail.
How can you use jails on your benefit? Ever tried to configure some new software and failed miserably? You can have three different jails running at the same time with different configurations. Want to try a new configuration in a production piece of hardware without applying it on the final users? You can do that with a small jail while the production environment is on in another bigger, chunkier jail.
Want to divide the hardware as a replica of the division of the team/s you are working with? Want to sell virtual machines with bare metal performance? Do you want to isolate some piece of critical software or even data in a more controlled environment? Do you have different clients and you want to use the same hardware but you want to avoid them seeing each other at the same time you maintain performance and reliability?
Are you a developer and you have to have reliable and portable snapshots of your work? Do you want to try new options-designs without breaking your previous work, in a timeless fashion? You can work on something, clone the jail and apply the new ideas on the project in a matter of seconds. You can stop there, export the filesystem snapshot containing all the environment and all your work and place it on a thumbdrive to later import it on a big production system. Want to change that image properties such as the network stack interface and ip? This is just one command away from you.
But what properties can you assign to a jail and how can I manage them you may be wondering. Hostname, disk quota, i/o, memory, cpu limits, network isolation, network virtualization, snapshots and the manage of those, migration and root privilege isolation to name a few. You can also clone them and import and export them between different systems. Some of these things because of ZFS. Iocage is a python program to manage jails and it takes profit from ZFS advantages.
But FreeBSD is not Linux you may say. No it is not. There are no run levels. The systemd factor is out of this equation. This is so since the begginning. Ever wondered where did vi come from? The TCP/IP stack? Your beloved macOS from Apple? All this is coming from the FreeBSD project. If you are used to Linux your adaptation period with any BSD will be short, very short. You will almost feel at home. Used to packaged software using yum or apt-get? No worries. With pkgng, the package management tool used in FreeBSD has almost 27.000 compiled packages for you to use. Almost all software found on any of the important GNU/Linux distros can be found here. Java, Python, C, C++, Clang, GCC, Javascript frameworks, Ruby, PHP, MySQL and the major forks, etc. All this opensource software, and much more, is available at your fingertips.
I am a developer and… frankly my time is money and I appreciate both much more than dealing with systems configuration, etc. You can set a VM using VMWare or VirtualBox and play with barebones FreeBSD or you can use TrueOS (a derivative) which comes in a server version and a desktop oriented one. The latter will be easier for you to play with. You may be doing this already with Linux. There is a third and very sensible option. FreeNAS, developed by iXSystems. It is FreeBSD based and offers all these technologies with a GUI. VMWare, Hyper-V? Nowadays you can get your hands off the CLI and get a decent, usable, nice GUI.
You say you play on the cloud. The major players already include FreeBSD in their offerings. You can find it in Amazon AWS or Azure (with official Microsoft support contracts too!). You can also find it in DigitalOcean and other hosting providers. There is no excuse. You can use it at home, at the office, with old or new hardware and in the cloud as well. You can even pay for a support contract to use it. Joyent, the developers of SmartOS have their own cloud with different locations around the globe. Have a look on them too.
If you want the original of ZFS and zones you may think of Solaris. But it’s fading away. But it really isn’t. When Oracle bouth Sun many people ran away in an stampide fashion. Some of the good folks working at Sun founded new projects. One of these is Illumos. Joyent is a company formed by people who developed these technologies. They are a cloud operator, have been recently bought by Samsung and have a very competent team of people providing great tech solutions. They have developed an OS, called SmartOS (based on Illumos) with all these features. The source from this goes back to the early days of UNIX. Do you remember the days of OpenSolaris when Sun opensourced the crown jewels? There you have it. A modern opensource UNIX operating system with the roots in their original place and the head planted on today’s needs.
In conclusion. If you are on GNU/Linux and you only use opensource software you may be doing it wrong. And missing goodies you may need and like. Once you put your hands on them, trust me, you won’t look back. And if you have some “old fashioned” admins who know Solaris, you can bring them to a new profitable and exciting life with both systems.
Still not convinced? Would you have ever imagined Microsoft supporting Linux? Even loving it? They do love now FreeBSD. And not only that, they provide their own image in the Azure Cloud and you can get Microsoft support, payed support if you want to use the platform on Azure. Ain’t it… surprising? Convincing at all?
PS: I haven’t mentioned both softwares, FreeBSD and SmartOS do have a Linux translation layer. This means you can run Linux binaries on them and the program won’t cough at all. Since the ABI stays stable the only thing you need to run a Linux binary is a translation between the different system calls and the libraries. Remember POSIX? Choose your poison and enjoy it.

###A partly-cloudy IPsec VPN

  • Audience

I’m assuming that readers have at least a basic knowledge of TCP/IP networking and some UNIX or UNIX-like systems, but not necessarily OpenBSD or FreeBSD. This post will therefore be light on details that aren’t OS specific and are likely to be encountered in normal use (e.g., how to use vi or another text editor.) For more information on these topics, read Absolute FreeBSD (3ed.) by Michael W. Lucas.

  • Overview

I’m redoing my DigitalOcean virtual machines (which they call droplets). My requirements are:

  • VPN
  • Road-warrior access, so I can use private network resources from anywhere.
  • A site-to-site VPN, extending my home network to my VPSes.
  • Hosting for public and private network services.
  • A proxy service to provide a public IP address to services hosted at home.

The last item is on the list because I don’t actually have a public IP address at home; my firewall’s external address is in the RFC 1918 space, and the entire apartment building shares a single public IPv4 address.1 (IPv6? Don’t I wish.) The end-state network will include one OpenBSD droplet providing firewall, router, and VPN services; and one FreeBSD droplet hosting multiple jailed services.
I’ll be providing access via these droplets to a NextCloud instance at home. A simple NAT on the DO router droplet isn’t going to work, because packets going from home to the internet would exit through the apartment building’s connection and not through the VPN. It’s possible that I could do work around this issue with packet tagging using the pf firewall, but HAProxy is simple to configure and unlikely to result in hard-to-debug problems. relayd is also an option, but doesn’t have the TLS parsing abilities of HAProxy, which I’ll be using later on.
Since this system includes jails running on a VPS, and they’ve got RFC 1918 addresses, I want them reachable from my home network. Once that’s done, I can access the private address space from anywhere through a VPN connection to the cloudy router.
The VPN itself will be of the IPsec variety. IPsec is the traditional enterprise VPN standard, and is even used for classified applications, but has a (somewhat-deserved) reputation for complexity, but recent versions of OpenBSD turn down the difficulty by quite a bit.

This VPN both separates internal network traffic from public traffic and uses encryption to prevent interception or tampering.
Once traffic has been encrypted, decrypting it without the key would, as Bruce Schneier once put it, require a computer built from something other than matter that occupies something other than space. Dyson spheres and a frakton of causality violation would possibly work, as would mathemagical technology that alters the local calendar such that P=NP.2 Black-bag jobs and/or suborning cloud provider employees doesn’t quite have that guarantee of impossibility, however. If you have serious security requirements, you’ll need to do better than a random blog entry.

##News Roundup
KLEAK: Practical Kernel Memory Disclosure Detection

Modern operating systems such as NetBSD, macOS, and Windows isolate their kernel from userspace programs to increase fault tolerance and to protect against malicious manipulations [10]. User space programs have to call into the kernel to request resources, via system calls or ioctls. This communication between user space and kernel space crosses a security boundary. Kernel memory disclosures - also known as kernel information leaks - denote the inadvertent copying of uninitialized bytes from kernel space to user space. Such disclosed memory may contain cryptographic keys, information about the kernel memory layout, or other forms of secret data. Even though kernel memory disclosures do not allow direct exploitation of a system, they lay the ground for it.
We introduce KLEAK, a simple approach to dynamically detect kernel information leaks. Simply said, KLEAK utilizes a rudimentary form of taint tracking: it taints kernel memory with marker values, lets the data travel through the kernel and scans the buffers exchanged between the kernel and the user space for these marker values. By using compiler instrumentation and rotating the markers at regular intervals, KLEAK significantly reduces the number of false positives, and is able to yield relevant results with little effort.
Our approach is practically feasible as we prove with an implementation for the NetBSD kernel. A small performance penalty is introduced, but the system remains usable. In addition to implementing KLEAK in the NetBSD kernel, we applied our approach to FreeBSD 11.2. In total, we detected 21 previously unknown kernel memory disclosures in NetBSD-current and FreeBSD 11.2, which were fixed subsequently. As a follow-up, the projects’ developers manually audited related kernel areas and identified dozens of other kernel memory disclosures.
The remainder of this paper is structured as follows. Section II discusses the bug class of kernel memory disclosures. Section III presents KLEAK to dynamically detect instances of this bug class. Section IV discusses the results of applying KLEAK to NetBSD-current and FreeBSD 11.2. Section V reviews prior research. Finally, Section VI concludes this paper.

###How To Create Official Synth Repo

  • System Environment

  • Make sure /usr/dports is updated and that it contains no cruft (git pull; git status). Remove any cruft.

  • Make sure your ‘synth’ is up-to-date ‘pkg upgrade synth’. If you already updated your system you may have to build synth from scratch, from /usr/dports/ports-mgmt/synth.

  • Make sure /etc/make.conf is clean.

  • Update /usr/src to the current master, make sure there is no cruft in it

  • Do a full buildworld, buildkernel, installkernel and installworld

  • Reboot

  • After the reboot, before proceeding, run ‘uname -a’ and make sure you are now on the desired release or development kernel.

  • Synth Environment

  • /usr/local/etc/synth/ contains the synth configuration. It should contain a synth.ini file (you may have to rename the template), and you will have to create or edit a LiveSystem-make.conf file.

  • System requirements are hefty. Just linking chromium alone eats at least 30GB, for example. Concurrent c++ compiles can eat up to 2GB per process. We recommend at least 100GB of SSD based swap space and 300GB of free space on the filesystem.

  • synth.ini should contain this. Plus modify the builders and jobs to suit your system. With 128G of ram, 30/30 or 40/25 works well. If you have 32G of ram, maybe 8/8 or less.

; Take care when hand editing!

[Global Configuration]
profileselected= LiveSystem

[LiveSystem]
Operatingsystem= DragonFly
Directorypackages= /build/synth/livepackages
Directoryrepository= /build/synth/livepackages/All
Directoryportsdir= /build/synth/dports
Directoryoptions= /build/synth/options
Directorydistfiles= /usr/distfiles
Directorybuildbase= /build/synth/build
Directorylogs= /build/synth/logs
Directoryccache= disabled
Directorysystem= /
Numberofbuilders= 30
Maxjobsperbuilder= 30
Tmpfsworkdir= true
Tmpfslocalbase= true
Displaywithncurses= true
leverageprebuilt= false

  • LiveSystem-make.conf should contain one line to restrict licensing to only what is allowed to be built as a binary package:

LICENSESACCEPTED= NONE

  • Make sure there is no other cruft in /usr/local/etc/synth/

  • In the example above, the synth working dirs are in “/build/synth”. Make sure the base directories exist. Clean out any cruft for a fresh build from-scratch:

rm -rf /build/synth/livepackages/*
rm -rf /build/synth/logs
mkdir /build/synth/logs

  • Run synth everything. I recommend doing this in a ‘screen’ session in case you lose your ssh session (assuming you are ssh’d into the build machine).

(optionally start a screen session)
synth everything

  • A full synth build takes over 24 hours to run on a 48-core box, around 12 hours to run on a 64-core box. On a 4-core/8-thread box it will take at least 3 days. There will be times when swap space is heavily used. If you have not run synth before, monitor your memory and swap loads to make sure you have configured the jobs properly. If you are overloading the system, you may have to ^C the synth run, reduce the jobs, and start it again. It will pick up where it left off.
  • When synth finishes, let it rebuild the database. You then have a working binary repo.
  • It is usually a good idea to run synth several times to pick up any stuff it couldn’t build the first time. Each of these incremental runs may take a few hours, depending on what it tries to build.

###Interview with founder and maintainer of GhostBSD, Eric Turgeon

  • Thanks you Eric for taking part. To start off, could you tell us a little about yourself, just a bit of background?
  • How did you become interested in open source?
  • When and how did you get interested in the BSD operating systems?
  • On your Twitter profile, you state that you are an automation engineer at iXsystems. Can you share what you do in your day-to-day job?
  • You are the founder and project lead of GhostBSD. Could you describe GhostBSD to those who have never used it or never heard of it?
  • Developing an operating system is not a small thing. What made you decide to start the GhostBSD project and not join another “desktop FreeBSD” related project, such as PC-BSD and DesktopBSD at the time?
  • How did you get to the name GhostBSD? Did you consider any other names?
  • You recently released GhostBSD 18.10? What’s new in that version and what are the key features? What has changed since GhostBSD 11.1?
  • The current version is 18.10. Will the next version be 19.04 (like Ubuntu’s version numbering), or is a new version released after the next stable TrueOS release
  • Can you tell us something about the development team? Is it yourself, or are there other core team members? I think I saw two other developers on your Github project page.
  • How about the relationship with the community? Is it possible for a community member to contribute, and how are those contributions handled?
  • What was the biggest challenge during development?
  • If you had to pick one feature readers should check out in GhostBSD, what is it and why?
  • What is the relationship between iXsystems and the GhostBSD project? Or is GhostBSD a hobby project that you run separately from your work at iXsystems?
  • What is the relationship between GhostBSD and TrueOS? Is GhostBSD TrueOS with the MATE desktop on top, or are there other modifications, additions, and differences?
  • Where does GhostBSD go from here? What are your plans for 2019?
  • Is there anything else that wasn’t asked or that you want to share?

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 13 Dec 2018 01:15:00 -0800
Episode 275: OpenBSD in Stereo | BSD Now 275
DragonflyBSD 5.4 has been released, down the Gopher hole with OpenBSD, OpenBSD in stereo with VFIO, BSD/OS the best candidate for legally tested open source Unix, OpenBGPD adds diversity to the routing server landscape, and more. Headlines DragonflyBSD 5.4 released DragonFly version 5.4 brings a new system compiler in GCC 8, improved NUMA support, a large of number network and virtual machine driver updates, and updates to video support. This release is 64-bit only, as with previous releases. The details of all commits between the 5.2 and 5.4 branches are available in the associated commit messages for 5.4.0rc and 5.4.0.
  • Big-ticket items
  • Much better support for asymmetric NUMA (Non-Uniform Memory Access) configurations. In particular, both the memory subsystem and the scheduler now understand the Threadripper 2990WX’s architecture. The scheduler will prioritize CPU nodes with direct-attached memory and the memory subsystem will normalize memory queues for CPU nodes without direct-attached memory (which improves cache locality on those CPUs).
  • Incremental performance work. DragonFly as a whole is very SMP friendly. The type of performance work we are doing now mostly revolves around improving fairness for shared-vs-exclusive lock clashes, reducing cache ping-ponging due to non-contending SMP locks (i.e. massive use of shared locks on shared resources), and so forth.
  • Major updates to dports brings us to within a week or two of FreeBSD’s ports as of this writing, in particular major updates to chromium, and making the whole mess work with gcc-8.
  • Major rewriting of the tty clist code and the tty locking code, significantly improving concurrency across multiple ttys and ptys.
  • GCC 8
  • DragonFly now ships with GCC 8.0, and runs as the default compiler. It is also now used for building dports.
  • GCC 4.7.4 and GCC 5.4.1 are still installed. 4.7.4 is our backup compiler, and 5.4.1 is still there to ensure a smooth transition, but should generally not be used. buildworld builds all three by default to ensure maximum compatibility.
  • Many passes through world sources were made to address various warnings and errors the new GCC brought with it.
  • HAMMER2
  • HAMMER2 is recommended as the default root filesystem in non-clustered mode.
  • Clustered support is not yet available.
  • Increased bulkfree cache to reduce the number of iterations required.
  • Fixed numerous bugs.
  • Improved support on low-memory machines.
  • Significant pre-work on the XOP API to help support future networked operations.
  • Details
  • Checksums MD5 (dfly-x86_64-5.4.0_REL.img) = 7277d7cffc92837c7d1c5dd11a11b98f MD5 (dfly-x86_64-5.4.0_REL.iso) = 6da7abf036fe9267479837b3c3078408 MD5 (dfly-x86_64-5.4.0_REL.img.bz2) = a77a072c864f4b72fd56b4250c983ff1 MD5 (dfly-x86_64-5.4.0_REL.iso.bz2) = 4dbfec6ccfc1d59c5049455db914d499
  • Downloads Links
DragonFly BSD is 64-bit only, as announced during the 3.8 release.
  • USB: dfly-x86_64-5.4.0_REL.img as bzip2 file
  • ISO: dfly-x86_64-5.4.0_REL.iso as bzip2 file
  • Uncompressed ISO: dfly-x86_64-5.4.0_REL.iso (For use with VPS providers as an install image.)
Down the Gopher hole with OpenBSD, Gophernicus, and TLS In the early 2000s I thought I had seen the worst of the web - Java applets, Macromedia (>Adobe) Flash, animated GIFs, javascript snow that kept you warm in the winter by burning out your CPU, and so on. For a time we learned from these mistakes, and started putting the burden on the server-side - then with improvements in javascript engines we started abusing it again with JSON/AJAX and it all went down hill from there. Like cloud computing, blockchains, machine learning and a tonne of other a la mode technologies around today - most users and service providers don’t need websites that consume 1GB of memory processing JS and downloading 50MB of compressed data just to read Alice’s one-page travel blog or Bob’s notes on porting NetBSD to his blood-pressure monitor. Before the HTTP web we relied on Prestel/Minitel style systems, BBS systems, and arguably the most accessible of all - Gopher! Gopher was similar to the locally accessed AmigaGuide format, in that it allowed users to search and retrieve documents interactively, with links and cross-references. Its efficiency and distraction-free nature make it attractive to those who are tired of the invasive, clickbait, ad-filled, javascript-laden web2/3.x. But enough complaining and evangelism - here’s how to get your own Gopher Hole! Gophernicus is a modern gopher daemon which aims to be secure (although it still uses inetd -_-); it’s even in OpenBSD ports so at least we can rely on it to be reasonably audited. If you need a starting point with Gopher, SDF-EU’s wiki has a good article here. Finally, if you don’t like gopher(1) - there’s always lynx(1) or NCSA Mosaic! I’ve added TLS support to Gophernicus so you don’t need to use stunnel anymore. The code is ugly and unpolished though so I wouldn’t recommend for production use. News Roundup OpenBSD in Stereo with Linux VFIO I use a Huawei Matebook X as my primary OpenBSD laptop and one aspect of its hardware support has always been lacking: audio never played out of the right-side speaker. The speaker did actually work, but only in Windows and only after the Realtek Dolby Atmos audio driver from Huawei was installed. Under OpenBSD and Linux, and even Windows with the default Intel sound driver, audio only ever played out of the left speaker. Now, after some extensive reverse engineering and debugging with the help of VFIO on Linux, I finally have audio playing out of both speakers on OpenBSD.
  • VFIO
The Linux kernel has functionality called VFIO which enables direct access to a physical device (like a PCI card) from userspace, usually passing it to an emulator like QEMU. To my surprise, these days, it seems to be primarily by gamers who boot Linux, then use QEMU to run a game in Windows and use VFIO to pass the computer’s GPU device through to Windows. By using Linux and VFIO, I was able to boot Windows 10 inside of QEMU and pass my laptop’s PCI audio device through to Windows, allowing the Realtek audio drivers to natively control the audio device. Combined with QEMU’s tracing functionality, I was able to get a log of all PCI I/O between Windows and the PCI audio device.
  • Using VFIO
To use VFIO to pass-through a PCI device, it first needs to be stubbed out so the Linux kernel’s default drivers don’t attach to it. GRUB can be configured to instruct the kernel to ignore the PCI audio device (8086:9d71) and explicitly enable the Intel IOMMU driver by adding the following to /etc/default/grub and running update-grub With the audio device stubbed out, a new VFIO device can be created from it Then the VFIO device (00:1f.3) can be passed to QEMU I was using my own build of QEMU for this, due to some custom logging I needed (more on that later), but the default QEMU package should work fine. The events.txt was a file of all VFIO events I wanted logged (which was all of them). Since I was frequently killing QEMU and restarting it, Windows 10 wanted to go through its unexpected shutdown routine each time (and would sometimes just fail to boot again). To avoid this and to get a consistent set of logs each time, I used qemu-img to take a snapshot of a base image first, then boot QEMU with that snapshot. The snapshot just gets thrown away the next time qemu-img is run and Windows always starts from a consistent state. QEMU will now log each VFIO event which gets saved to a debug-output file. With a full log of all PCI I/O activity from Windows, I compared it to the output from OpenBSD and tried to find the magic register writes that enabled the second speaker. After days of combing through the logs and annotating them by looking up hex values in the documentation, diffing runtime register values, and even brute-forcing it by mechanically duplicating all PCI I/O activity in the OpenBSD driver, nothing would activate the right speaker. One strange thing that I noticed was if I booted Windows 10 in QEMU and it activated the speaker, then booted OpenBSD in QEMU without resetting the PCI device’s power in-between (as a normal system reboot would do), both speakers worked in OpenBSD and the configuration that the HDA controller presented was different, even without any changes in OpenBSD. A Primer on Intel HDA Most modern computers with integrated sound chips use an Intel High Definition Audio (HDA) Controller device, with one or more codecs (like the Realtek ALC269) hanging off of it. These codecs do the actual audio processing and communicate with DACs and ADCs to send digital audio to the connected speakers, or read analog audio from a microphone and convert it to a digital input stream. In my Huawei Matebook X, this is done through a Realtek ALC298 codec. On OpenBSD, these HDA controllers are supported by the azalia(4) driver, with all of the per-codec details in the lengthy azalia_codec.c file. This file has grown quite large with lots of codec- and machine-specific quirks to route things properly, toggle various GPIO pins, and unmute speakers that are for some reason muted by default. The azalia driver talks to the HDA controller and sets up various buffers and then walks the list of codecs. Each codec supports a number of widget nodes which can be interconnected in various ways. Some of these nodes can be reconfigured on the fly to do things like turning a microphone port into a headphone port. The newer Huawei Matebook X Pro released a few months ago is also plagued with this speaker problem, although it has four speakers and only two work by default. A fix is being proposed for the Linux kernel which just reconfigures those widget pins in the Intel HDA driver. Unfortunately no pin reconfiguration is enough to fix my Matebook X with its two speakers. While reading more documentation on the HDA, I realized there was a lot more activity going on than I was able to see through the PCI tracing. For speed and efficiency, HDA controllers use a DMA engine to transfer audio streams as well as the commands from the OS driver to the codecs. In the output above, the CORBWP=0; size=256 and RIRBRP=0, size=256 indicate the setup of the CORB (Command Output Ring Buffer) and RIRB (Response Input Ring Buffer) each with 256 entries. The HDA driver allocates a DMA address and then writes it to the two CORBLBASE and CORBUBASE registers, and again for the RIRB. When the driver wants to send a command to a codec, such as CORB_GET_PARAMETER with a parameter of COP_VOLUME_KNOB_CAPABILITIES, it encodes the codec address, the node index, the command verb, and the parameter, and then writes that value to the CORB ring at the address it set up with the controller at initialization time (CORBLBASE/CORBUBASE) plus the offset of the ring index. Once the command is on the ring, it does a PCI write to the CORBWP register, advancing it by one. This lets the controller know a new command is queued, which it then acts on and writes the response value on the RIRB ring at the same position as the command (but at the RIRB’s DMA address). It then generates an interrupt, telling the driver to read the new RIRBWP value and process the new results. Since the actual command contents and responses are handled through DMA writes and reads, these important values weren’t showing up in the VFIO PCI trace output that I had gathered. Time to hack QEMU.
  • Logging DMA Memory Values in QEMU
Since DMA activity wouldn’t show up through QEMU’s VFIO tracing and I obviously couldn’t get Windows to dump these values like I could in OpenBSD, I could make QEMU recognize the PCI write to the CORBWP register as an indication that a command has just been written to the CORB ring. My custom hack in QEMU adds some HDA awareness to remember the CORB and RIRB DMA addresses as they get programmed in the controller. Then any time a PCI write to the CORBWP register is done, QEMU fetches the new CORB command from DMA memory, decodes it into the codec address, node address, command, and parameter, and prints it out. When a PCI read of the RIRBWP register is requested, QEMU reads the response and prints the corresponding CORB command that it stored earlier. With this hack in place, I now had a full log of all CORB commands and RIRB responses sent to and read from the codec: An early version of this patch left me stumped for a few days because, even after submitting all of the same CORB commands in OpenBSD, the second speaker still didn’t work. It wasn’t until re-reading the HDA spec that I realized the Windows driver was submitting more than one command at a time, writing multiple CORB entries and writing a CORBWP value that was advanced by two. This required turning my CORB/RIRB reading into a for loop, reading each new command and response between the new CORBWP/RIRBWP value and the one previously seen. Sure enough, the magic commands to enable the second speaker were sent in these periods where it submitted more than one command at a time.
  • Minimizing the Magic
The full log of VFIO PCI activity from the Windows driver was over 65,000 lines and contained 3,150 CORB commands, which is a lot to sort through. It took me a couple more days to reduce that down to a small subset that was actually required to activate the second speaker, and that could only be done through trial and error:
  • Boot OpenBSD with the full list of CORB commands in the azalia driver
  • Comment out a group of them
  • Compile kernel and install it, halt the QEMU guest
  • Suspend and wake the laptop, resetting PCI power to the audio device to reset the speaker/Dolby initialization and ensure the previous run isn’t influencing the current test (I’m guessing there is an easier to way to reset PCI power than suspending the laptop, but oh well)
  • Start QEMU, boot OpenBSD with the new kernel
  • Play an MP3 with mpg123 which has alternating left- and right-channel audio and listen for both channels to play
This required a dozen or so iterations because sometimes I’d comment out too many commands and the right speaker would stop working. Other times the combination of commands would hang the controller and it wouldn’t process any further commands. At one point the combination of commands actually flipped the channels around so the right channel audio was playing through the left speaker.
  • The Result
After about a week of this routine, I ended up with a list of 662 CORB commands that are needed to get the second speaker working. Based on the number of repeated-but-slightly-different values written with the 0x500 and 0x400 commands, I’m guessing this is some kind of training data and that this is doing the full Dolby/Atmos system initialization, not just turning on the second speaker, but I could be completely wrong. In any case, the stereo sound from OpenBSD is wonderful now and I can finally stop downmixing everything to mono to play from the left speaker. In case you ever need to do this, sndiod can be run with -c 0:0 to reduce the channels to one. Due to the massive size of the code needed for this quirk, I’m not sure if I’ll be committing it upstream in OpenBSD or just saving it for my own tree. But at least now the hardware support chart for my Matebook is all yeses for the things I care about. I’ve also updated the Linux bug report that I opened before venturing down this path, hoping one of the maintainers of that HDA code that works at Intel or Realtek knew of a solution I could just port to OpenBSD. I’m curious to see what they’ll do with it. Why BSD/OS is the best candidate for being the only tested legally open UNIX
  • Introduction
The UNIX® system is an old operating system, possibly older than many of the readers of this post. However, despite its age, it still has not been open sourced completely. In this post, I will try to detail which parts of which UNIX systems have not yet been open sourced. I will focus on the legal situation in Germany in particular, taking it representative of European law in general – albeit that is a stretch, knowing the diversity of European jurisdictions. Please note that familiarity with basic terms of copyright law is assumed.
  • Ancient UNIX
The term “Ancient UNIX” refers to the versions of UNIX up to and including Seventh Edition UNIX (1979) including the 32V port to the VAX. Ancient UNIX was created at Bell Laboratories, a subsidiary of AT&T at the time. It was later transferred of the AT&T UNIX Support Group, then AT&T Information Systems and finally the AT&T subsidiary UNIX System Laboratories, Inc. (USL). The legal situation differs between the United States of America and Germany. In a ruling as part of the UNIX System Laboratories, Inc. v. Berkeley Software Design, Inc. (USL v. BSDi) case, a U.S. court found that USL had no copyright to the Seventh Edition UNIX system and 32V – arguably, by extension, all earlier versions of Ancient UNIX as well – because USL/AT&T had failed to affix copyright notices and could not demonstrate a trade secret. Due to the obsessive tendency of U.S. courts to consider themselves bound to precedents (cf. the infamous Pierson v. Post case), it can be reasonably expected that this ruling would be honored and applied in subsequent cases. Thus under U.S. law, Ancient UNIX can be safely assumed to belong in the public domain. The situation differs in Germany. Unlike the U.S., copyright never needed registration in order to exist. Computer programs are works in the sense of the German 1965 Act on Copyright and Related Rights (Copyright Act, henceforth CopyA) as per CopyA § 2(1) no. 1. Even prior to the amendment of CopyA § 2(1) to include computer programs, computer programs have been recognized as copyrightable works by the German Supreme Court (BGHZ 112, 264 Betriebssystem, no. 19); CopyA § 137d(1) rightly clarifies that. The copyright holder at 1979 would still have been USL via Bell Labs and AT&T. Copyright of computer programs is transferred to the employer upon creation under CopyA § 69(1). Note that this does not affect expiry (Daniel Kaboth/Benjamin Spies, commentary on CopyA §§ 69a‒69g, in: Hartwig Ahlberg/Horst-Peter Götting (eds.), Urheberrecht: UrhG, KUG, VerlG, VGG, Kommentar, 4th ed., C. H. Beck, 2018, no. 16 ad CopyA § 69b; cf. Bundestag-Drucksache [BT-Drs.] 12/4022, p. 10). Expiry occurs 70 years after the death of the (co-)author that died most recently as per CopyA § 65(1) and 64; this has been the case since at least the 1960s, meaning there is no way for copyright to have expired already (old version, as per Bundesgesetzblatt Part I No. 51 of September 16, 1965, pp. 1273‒1294). In Germany, private international law applies the so-called “Territorialitätsprinzip” for intellectual property rights. This means that the effect of an intellectual property right is limited to the territory of a state (Anne Lauber-Rönsberg, KollisionsR, in: Hartwig Ahlberg/Horst-Peter Götting (eds.), ibid., pp. 2241 et seqq., no. 4). Additionally, the “Schutzlandprinzip” applies; this means that protection of intellectual property follows the lex loci protectionis, i.e. the law of the country for which protection is sought (BGH GRUR 2015, 264 HiHotel II, no. 25; BGH GRUR 2003, 328 Sender Felsberg, no. 24), albeit this is criticized in parts of doctrine (Lauber-Rönsberg, ibid., no. 10). The “Schutzlandprinzip” requires that the existence of an intellectual property right be verified as well (BGH ZUM 2016, 522 Wagenfeld-Leuchte II, no. 19). Thus, in Germany, copyright on Ancient UNIX is still alive and well. Who has it, though? A ruling by the U.S. Court of Appeals, Tenth Circuit, in the case of The SCO Group, Inc. v. Novell, Inc. (SCO v. Novell) in the U.S. made clear that Novell owns the rights to System V – thus presumably UNIX System III as well – and Ancient UNIX, though SCO acquired enough rights to develop UnixWare/OpenServer (Ruling 10-4122 [D.C. No. 2:04-CV-00139-TS], pp. 19 et seq.). Novell itself was purchased by the Attachmate Group, which was in turn acquired by the COBOL vendor Micro Focus. Therefore, the rights to SVRX and – outside the U.S. – are with Micro Focus right now. If all you care about is the U.S., you can stop reading about Ancient UNIX here. So how does the Caldera license factor into all of this? For some context, the license was issued January 23, 2002 and covers Ancient UNIX (V1 through V7 including 32V), specifically excluding System III and System V. Caldera, Inc. was founded in 1994. The Santa Cruz Operation, Inc. sold its rights to UNIX to Caldera in 2001, renamed itself to Tarantella Inc. and Caldera renamed itself The SCO Group. Nemo plus iuris ad alium transferre potest quam ipse habet; no one can transfer more rights than he has. The question now becomes whether Caldera had the rights to issue the Caldera license. I’ve noted it above but it needs restating: Foreign decisions are not necessarily accepted in Germany due to the “Territorialitätsprinzip” and “Schutzlandprinzip” – however, I will be citing a U.S. ruling for its assessment of the facts for the sake of simplicity. As per ruling 10-4122, “The district court found the parties intended for SCO to serve as Novell’s agent with respect to the old SVRX licenses and the only portion of the UNIX business transferred outright under the APA [asset purchase agreement] was the ability to exploit and further develop the newer UnixWare system. SCO was able to protect that business because it was able to copyright its own improvements to the system. The only reason to protect the earlier UNIX code would be to protect the existing SVRX licenses, and the court concluded Novell retained ultimate control over that portion of the business under the APA.” The relevant agreements consist of multiple pieces: the base Asset Purchase Agreement “APA” (Part I) the base Asset Purchase Agreement “APA” (Part II) the Operating Agremeent and Amendment 1 to the APA the Amendment 2 to the APA The APA dates September 19, 1995, from before the Caldera license. Caldera cannot possibly have acquired rights that The Santa Cruz Operation, Inc. itself never had. Furthermore, I’ve failed to find any mention of Ancient UNIX; all that is transferred is rights to SVRX. Overall, I believe that the U.S. courts’ assesment of the facts represents the situation accurately. Thus for all intents and purposes, UNIX up to and including System V remained with Novell/Attachmate/Micro Focus. Caldera therefore never had any rights to Ancient UNIX, which means it never had the rights to issue the Caldera license. The Caldera license is null and void – in the U.S. because the copyright has been lost due to formalities, everywhere else because Caldera never had the rights to issue it. The first step to truly freeing UNIX would this be to get Micro Focus to re-issue the Caldera license for Ancient UNIX, ideally it would now also include System III and System V.
  • BSD/OS
Another operating system near UNIX is of interest. The USL v. BSDi lawsuit includes two parties: USL, which we have seen above, and Berkeley Software Design, Inc. BSDi sold BSD/386 (later BSD/OS), which was a derivative of 4.4BSD. The software parts of the BSDi company were acquired by Wind River Systems, whereas the hardware parts went to iXsystems. Copyright is not disputed there, though Wind River Systems ceased selling BSD/OS products 15 years ago, in 2003. In addition, Wind River System let their trademark on BSD expire, though this is without consequence for copyright. BSD/OS is notable in the sense that it powered much of early internet infrastructure. Traces of its legacy can still be found on Richard Stevens’ FAQ. To truly make UNIX history free, BSD/OS would arguably also need to see a source code release. BSD/OS at least in its earliest releases under BSDi would ship with source code, though under a non-free license, far from BSD or even GPL licensing.
  • System V
The fate of System V as a whole is difficult to determine. Various licenses have been granted to a number of vendors (Dell UNIX comes to mind; HP for HP-UX, IBM for AIX, SGI UNIX, etc.). Sun released OpenSolaris – notoriously, Oracle closed the source to Solaris again after its release –, which is a System V Release 4 descendant. However, this means nothing for the copyright or licensing status of System V itself. Presumably, the rights with System V still remain with Novell (now Micro Focus): SCO managed to sublicense rights to develop and sell UnixWare/OpenServer, themselves System V/III descendants, to unXis, Inc. (now known as Xinuos, Inc.), which implies that Xinuos is not the copyright holder of System V. Obviously, to free UNIX, System V and its entire family of descendants would also need to be open sourced. However, I expect tremendous resistance on part of all the companies mentioned. As noted in the “Ancient UNIX” section, Micro Focus alone would probably be sufficient to release System V, though this would mean nothing for the other commercial System V derivatives.
  • Newer Research UNIX
The fate of Bell Labs would be a different one; it would go on to be purchased by Lucent, now part of Nokia. After commercial UNIX got separated out to USL, Research UNIX would continue to exist inside of Bell Labs. Research UNIX V8, V9 and V10 were not quite released by Alcatel-Lucent USA Inc. and Nokia in 2017. However, this is merely a notice that the companies involved will not assert their copyrights only with respect to any non-commercial usage of the code. It is still not possible, over 30 years later, to freely use the V8 code.
  • Conclusion In the U.S., Ancient UNIX is freely available. People located everywhere else, however, are unable to legally obtain UNIX code for any of the systems mentioned above. The exception being BSD/OS, assuming a purchase of a legitimate copy of the source code CD. This is deeply unsatisfying and I implore all involved companies to consider open sourcing (preferably under a BSD-style license) their code older than a decade, if nothing else, then at least for the sake of historical purposes. I would like to encourage everybody reading this to consider reaching out to Micro Focus and Wind River Systems about System V and BSD/OS, respectively. Perhaps the masses can change their minds.
A small note about patents: Some technologies used in newer iterations of the UNIX system (in particular the System V derivatives) may be encumbered with software patents. An open source license will not help against patent infringement claims. However, the patents on anything used in the historical operating systems will certainly have expired by now. In addition, European readers can ignore this entirely – software patents just aren’t a thing. OpenBGPD - Adding Diversity to the Route Server Landscape
  • Introduction
As of last year, there was effectively only a single solution in the Route Server vendor market: the BIRD Internet routing daemon. NIC.CZ (the organisation developing BIRD) has done fantastic work on maintaining their BGP-4 implementation, however, it’s not healthy to have virtually every Internet Exchange Point (IXP) in the RIPE NCC service region depend on a single open source project. The current situation can be compared to the state of the DNS root nameservers back in 2002 - their dependence on the BIND nameserver daemon and the resulting development of NSD as an alternative by NLnet, in cooperation with the RIPE NCC. OpenBGPD used to be one of the most popular Route Server implementations until the early 2010s. OpenBGPD’s main problem was that its performance couldn’t keep up with the Internet’s growth, so it lost market share. An analysis by Job Snijders suggested that a modernised OpenBGPD distribution would be a most viable option to regain diversity on the Route Server level.
  • Missing features in OpenBGPD
The following main missing features were identified in OpenBGPD:
  • Performance
In previous versions of OpenBGPD, the filtering performance didn’t allow proper filtering of all EBGP sessions. Current best practice at IXP Route Servers is to carefully evaluate and validate of all routes learned from EBGP peers. The OpenBGPD ruleset required to do correct filtering (in many deployment scenarios) was simply too lengthy - and negatively impacted service performance during configuration reloads. While filtering performance is the biggest bottleneck, general improvements to the Routing Information Base were also made to improve scalability. IXP Route Servers with a few hundred peering sessions are commonplace and adding new sessions shouldn’t impact the Route Servers’ service to other peers. We found that performance was the most pressing issue that needed to be tackled.
  • Lack of RPKI Origin Validation
As we’ve seen, Internet operators are moving to adopt RPKI based BGP Origin Validation. While it was theoretically possible to emulate RFC 6811-style Origin Validation in previous versions of OpenBGPD, the required configuration wasn’t optimised for performance and wasn’t user friendly. We believe that BGP Origin Validation should be as easy as possible - this requires BGP-4 vendors to implement native, optimised routines for Origin Validation. Of course, enabling Origin Validation shouldn’t have an impact on performance either when processing BGP updates or when updating the Route Origin Authorisation (ROA) table itself.
  • Portability
OpenBGPD is an integral part of OpenBSD, but IXPs may prefer to run their services infrastructure on an operating system of their choice. Making sure that there’s a portable OpenBGPD version which follows the OpenBSD project release cycle will give IXPs this option.
  • Development steps
By addressing the issues mentioned above, we could bring back OpenBGPD as a viable Route Server implementation. Since I was one of the core OpenBGPD developers, I was asked if I wanted to pick up this project again. Thanks to the funding from the RIPE NCC Project Fund, this was possible. Starting in June 2018, I worked full time on this important community project. Over the last few months, many of the problems are already addressed and are now part of the OpenBSD 6.4 release. So far, 154 commits were made to OpenBGPD during the 6.4 development cycle - around 8% of all commits ever to OpenBGPD! This shows that due to funding and dedicated resources, a lot of work could be pushed into the latest release of OpenBGPD.
  • OpenBGPD 6.4
The OpenBGPD version, as part of OpenBSD 6.4 release, demonstrates great progress. Even though there have been many changes to the core of OpenBGPD, the released version is as solid and reliable as previous releases and the many bug fixes and improvements make this the best OpenBGPD release so far. The changes in the filter language allow users to write more efficient rulesets while the introduction of RPKI origination validation fixes an important missing feature. For IXPs, OpenBGPD now is an alternative again. There are still open issues, but the gap is closing!
  • Feature highlights
The following changes should be highlighted:
  • Introduction of background soft-reconfiguration on config reload. Running the soft-reconfiguration task in the background allows for new updates and withdraws to be processed at the same time. This improves convergence time - one of the key metrics for Route Servers.
  • BGP Origin Validation when a roa-set is configured Every EBGP route announcement is validated against the locally configured VRP table entries. Depending on the validation process’s outcome, the validation state is set to valid, invalid or not found. The filter language has been extended to allow checking for the origin validation state, and thanks to this, it is possible to deny invalid prefixes or regard valid prefixes different to the ones that aren’t found. The roa-set table is read from the configuration file and updated during configuration reloads. On production systems reloading the roa-set and applying it to all prefixes is done in a couple of seconds.
  • Fast prefix-set lookups In OpenBSD 6.3 prefix-sets got introduced in OpenBGPD. A prefix-set combines many prefix lookups into a single filter rule. The original implementation wasn’t optimised but now a fast trie lookup is used. Thanks to this, large IRR DB prefix tables can now be implemented efficiently.
  • Introduction of as-sets Similar to prefix-sets, as-sets help group many AS numbers into a single lookup. Thanks to this, large IRR DB origin AS tables can be implemented efficiently. Introduction of origin-sets
  • Looking at the configurations of Route Servers doing full filtering, it was noticed that a common lookup was binding a prefix to an origin AS - similar to how a roa-set is used for RPKI. These origin-set tables are used to extend the IRR prefix lookup and generated from alternative sources.
  • Improving third party tools
Users can only benefit from the changes introduced in OpenBGPD 6.4 when the surrounding 3rd party tools are adjusted accordingly. Two opensource projects such as bgpq3 and arouteserver are frequently used by network operators and IXPs to generate BGP configurations. Thanks to our contributions to those projects, we were able to get them ready for all the new features in OpenBGPD.
  • bgpq3 was extended to create as-set and prefix-set tables based on IRR DB entries. This is replacing the old way of doing the same with a large amount of filter rules. Thanks to the quick response from the bgpq3 maintainer, it was possible to ship OpenBSD 6.4 with a bgpq3 package that includes all the new features.
  • arouteserver was adjusted to implement RPKI roa-set, as-set, prefix-set, and origin-set to generate a much better-performing configurations for the 6.4 version. With the v0.20.0 release of arouteserver, IXPs are able to generate an OpenBGPD configuration which is a ton faster but also implements the new functionalities. Looking at YYCIX (the resident IXP in Calgary, Canada) the ruleset generated by arouteserver was reduced from 370,000 rules to well under 6,000 rules. This resulted in the initial convergence time dropping from over 1 hour to less than 2 minutes, and subsequent configuration reloads are hitless and no longer noticeable.
  • What still needs to be done
A sizeable chunk of work still left on the table is the rework of the RIB data structures in OpenBGPD - these haven’t been changed since the initial design of OpenBGPD in 2003. There’s currently ongoing work (in small steps, to avoid jeopardising the stability of OpenBGPD) to modernise these data-structures. The goal is to provide better decoupling of the filter step from storing RIB database changes, to pave the way to multi-threaded operations at a later point.
  • Looking forward
  • Job Snijders oversaw this year’s fundraising and project management, he adds:
It’s been incredibly productive to create an environment where a core developer is allowed to work full time on the OpenBGPD code base. However, it’s important to note there still is room for a number of new features to help improve its operational capabilities (such as BMP, RFC 7313, ADD_PATH, etc). It’d be beneficial to the Internet community at large if we can extend Claudio Jeker’s involvement for another year. Open source software doesn’t grow on trees! Strategic investments are the only way to keep OpenBGPD’s roadmap aligned with Internet growth and operator requirements. Beastie Bits Feedback/Questions
  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Sun, 09 Dec 2018 01:00:00 -0800
Episode 274: Language: Assembly | BSD Now 274
Assembly language on OpenBSD, using bhyve for FreeBSD development, FreeBSD Gaming, FreeBSD for Thanksgiving, no space left on Dragonfly’s hammer2, and more.

##Headlines
Assembly language on OpenBSD amd64+arm64

This is a short introduction to assembly language programming on OpenBSD/amd64+arm64. Because of security features in the kernel, I have had to rethink a series of tutorials covering Aarch64 assembly language on OpenBSD, and therefore this will serve as a placeholder-cum-reminder.

OpenBSD, like many UNIX and unix-like operating systems, now uses the Executable and Linkable Format (ELF) for its binary libraries and executables. Although the structure of this format is beyond the scope of this short introduction, it is necessary for me to explain part of one of the headers.

Within the program header there are sections known as PT_NOTE that OpenBSD and other systems use to distinguish their ELF executables - OpenBSD looks for this section to check if it should attempt to execute the program or not.

  • Our first program: in C!

It’s often a good idea to prototype your assembly programs in a high level language such as C - it can then double up as both a set of notes and a working program that you can debug and compile into assembly language to compare with your own asm code.

  • See the article for the rest on:
  • Our first program: in x86-64 Asm (AT&T/GAS syntax)
  • Our first program: in inline x86-64 assembly
  • Our first program: in x86-64 asm (NASM syntax)
  • Our first program: in ARMv8 AArch64 assembly

###Using bhyve for FreeBSD Development

  • The Hypervisor

The bhyve hypervisor requires a 64-bit x86 processor with hardware support for virtualization. This requirement allows for a simple, clean hypervisor implementation, but it does require a fairly recent
processor. The current hypervisor requires an Intel processor, but there is an active development branch with support for AMD processors.
The hypervisor itself contains both user and kernel components. The kernel driver is contained in the vmm.ko module and can be loaded either at boot from the boot loader or at runtime. It must
be loaded before any guests can be created. When a guest is created, the kernel driver creates a device file in /dev/vmm which is used by the user programs to interact with the guest.
The primary user component is the bhyve(8) program. It constructs the emulated device tree in the guest and provides the implementation for most of the emulated devices. It also calls the kernel driver to execute the guest. Note that the guest always executes inside the driver itself, so guest execution time in the host is counted as system time in the bhyve process.
Currently, bhyve does not provide a system firmware interface to the guest (neither BIOS nor UEFI). Instead, a user program running on the host is used to perform boot time operations including loading the guest operating system kernel into the guest’s memory and setting the initial guest state so that the guest begins execution at the kernel’s entry point. For FreeBSD guests, the bhyveload(8) program can be used to load the kernel and prepare the guest for execution. Support for some other operating systems is available via the grub2-bhyve program which is available via the sysutils/grub2-bhyve port or as a prebuilt package.
The bhyveload(8) program in FreeBSD 10.0 only supports 64-bit guests. Support for 32-bit guests will be included in FreeBSD 10.1.

  • See the article for the very technical breakdown of the following:

  • Network Setup

  • Bridged Configuration

  • Private Network with NAT

  • Using dnsmasq with a Private Network

  • Running Guests via vmrun.sh

  • Configuring Guests

  • Using a bhyve Guest as a Target

  • Conclusion

The bhyve hypervisor is a nice addition to a FreeBSD developer’s toolbox. Guests can be used both to develop new features and to test merges to stable branches. The hypervisor has a wide variety of uses beyond developing FreeBSD as well.

##News Roundup
Games on FreeBSD

What do all programmers like to do after work? Ok, what do most programers like to do after work? The answer is simple: play a good game! Recently at the Polish BSD User Group meetup mulander was telling us how you can play games on OpenBSD. Today let’s discuss how this looks in the FreeBSD world using the “server only” operating system.

  • XNA based games

One of the ways of playing natively is to play indie games which use XNA. XNA is a framework from Microsoft which uses .NET, for creating games. Fortunately, in the BSD world we have Mono, an open source implementation of Microsoft’s .NET Framework which you can use to run games. There is also FNA framework which is a reimplementation of XNA which allows you to run the games under Linux. Thomas Frohwein, from OpenBSD, prepared a script, fnaify. Fnaify translate all dependencies used by an FNA game to OpenBSD dependencies.
I decided to port the script to FreeBSD. The script is using /bin/sh which in the case of OpenBSD is a Korn Shell.

I didn’t test it with many games, but I don’t see any reason why it shouldn’t work with all the games tested by the OpenBSD guys. For example, with:

  • Cryptark

  • Rouge Legacy

  • Apotheon

  • Escape Goat

  • Bastion

  • CrossCode

  • Atom Zombie Smasher

  • Open-Source games

In FreeBSD and OpenBSD we also will find popular games which were open sourced. For example, I spend a lot of time playing in Quake 3 Arena on my FreeBSD machine. You can very simply install it using pkg: # pkg install ioquake3

Then move the files for the skins and maps to the .ioquake3 directory from your copy of Quake. In the past I also played UrbanTerror which is a fully open source shooter based on the Quake 3 Arena engine. It’s is also very easy to install it from ports: # pkg install iourbanterror

In the ports tree in the games directory you can find over 1000 directories, many of them with fully implemented games. I didn’t test many games in this category, but you can find some interesting titles like:

  • openxcom (Open-source re-implementation of the original X-Com)
  • openjazz (Free re-implementation of the Jazz Jackrabbit™ game engine)
  • corsixth (Open source re-implementation of Theme Hospital)
  • quake2
  • openra (Red Alert)
  • openrct2 (Open source re-implementation of RollerCoaster Tycoon 2)
  • openmw (Open source engine reimplementation of the game Morrowind)

All those titles are simply installed through the packages. In that case I don’t think FreeBSD has any difference from OpenBSD.

  • Wine

One of the big advantages of FreeBSD over OpenBSD is that FreeBSD supports wine. Wine allows you to run Windows applications under other operating systems (including mac). If you are a FreeBSD 11 user, you can simply fetch wine from packages: # pkg install i386-wine

To run Windows games, you need to have a 32-bit wine because most of the games on Windows are built on 32-bits (maybe this has changed – I don’t play so much these days). In my case, because I run FreeBSD-CURRENT I needed to build wine from ports. It wasn’t nice, but it also wasn’t unpleasant. The whole step-by-step building process of a wine from ports can be found here.

  • Summary

As you can see there are many titles available for *BSDs. Thanks to the FNA and fnaify, OpenBSD and FreeBSD can work with indie games which use the XNA framework. There are many interesting games implemented using this framework. Open source is not only for big server machines, and there are many re-implementations of popular games like Theme Hospital or RollerCoaster Tycoon 2. The biggest market is still enabled through wine, although its creates a lot of problems to run the games. Also, if you are an OpenBSD user only this option is not available for you. Please also note that we didn’t discuss any other emulators besides wine. In OpenBSD and FreeBSD there are many of them for GameBoy, SNES, NeoGeo and other games consoles.

###FreeBSD For Thanksgiving

I’ve been working on FreeBSD for Intel for almost 6 months now. In the world of programmers, I am considered an old dog, and these 6 months have been all about learning new tricks. Luckily, I’ve found myself in a remarkably inclusive and receptive community whose patience seems plentiful. As I get ready to take some time off for the holidays, and move into that retrospective time of year, I thought I’d beat the rush a bit and update on the progress
Earlier this year, I decided to move from architect of the Linux graphics driver into a more nebulous role of FreeBSD enabling. I was excited, but also uncertain if I was making the right decision.
Earlier this half, I decided some general work in power management was highly important and began working there. I attended BSDCam (handsome guy on the right), and led a session on Power Management. I was honored to be able to lead this kind of effort.
Earlier this quarter, I put the first round of my patches up for review, implementing suspend-to-idle. I have some rougher patches to handle s0ix support when suspending-to-idle. I gave a talk MeetBSD about our team’s work.
Earlier this month, I noticed that FreeBSD doesn’t have an implementation for Intel Speed Shift (HWPstates), and I started working on that.
Earlier this week, I was promoted from a lowly mentee committer to a full src committer.
Earlier today, I decided to relegate my Linux laptop to the role of my backup machine, and I am writing this from my Dell XPS13 running FreeBSD

vandamme 13.0-CURRENT FreeBSD 13.0-CURRENT #45 881fee072ff(hwp)-dirty: Mon Nov 19 16:19:32 PST 2018 bwidawsk@vandamme:/usr/home/bwidawsk/usr/obj/usr/home/bwidawsk/usr/src/amd64.amd64/sys/DEVMACHINE amd64

6 months later, I feel a lot less uncertain about making the right decision. In fact, I think both opportunities would be great, and I’m thankful this Thanksgiving that this is my life and career. I have more plans and things I want to get done. I’m looking forward to being thankful again next year.

###hammer2: no space left on device on Dragonfly BSD

  • The Issue

hammer2 does not actually delete a file when you rm or unlink it. Since recovery of the file is possible (this is the design of hammer2), there will still be an entry taking up data. It’s similar to how git works.
Even with 75% usage listed here, the filesystem could still have filled up. If you are using it as your root filesystem, then attempts to clean up data may fail. If the kernel panics over this, you will see something like this.

  • The Fix

If you have a recent enough version of the rescue ramdisk installed, on bootup you can press ‘r’ and access the rescue ramdisk. Your provider will have to offer some sort of remote interface for interacting with the operating system before it boots, like VNC or IPMI. You can then mount your filesystem using:

[root@ ~]# mkdir /tmp/fs
[root@ ~]# mount_hammer2 -o local /dev/vbd0s1a /tmp/fs

If you receive an error that /sbin/hammer2 is not found, then your rescue ramdisk is not up to date enough. In that scenario, download the latest 5.2 iso from dragonflybsd.org and boot from the cd-rom on your virtual machine or physical device. Just login as root instead of installer.
If the mount does succeed, then all you have to do is run the following twice:

[root@ ~]# /sbin/hammer2 bulkfree /tmp/fs

If you do not have enough memory on your machine, you may need to mount swap. Add your swap partition to the /etc/fstab and then do:

[root@ ~]# swapon -a

Once you have ran the bulkfree command twice, the usage reported by df -h will be correct. However, there is a chance on reboot that a core dump will be placed in /var/crash/ so be prepared to have plenty of space free in case that happens. You should also delete any files you can and run the bulkfree operation twice afterwards to clear up additional space.

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Wed, 28 Nov 2018 23:00:00 -0800
Episode 273: A Thoughtful Episode | BSD Now 273
Thoughts on NetBSD 8.0, Monitoring love for a GigaBit OpenBSD firewall, cat’s source history, X.org root permission bug, thoughts on OpenBSD as a desktop, and NomadBSD review.

##Headlines
Some thoughts on NetBSD 8.0

NetBSD is a highly portable operating system which can be run on dozens of different hardware architectures. The operating system’s clean and minimal design allow it to be run in all sorts of environments, ranging from embedded devices, to servers, to workstations. While the base operating system is minimal, NetBSD users have access to a large repository of binary packages and a ports tree which I will touch upon later.
I last tried NetBSD 7.0 about three years ago and decided it was time to test drive the operating system again. In the past three years NetBSD has introduced a few new features, many of them security enhancements. For example, NetBSD now supports write exclusive-or execute (W^X) protection and address space layout randomization (ASLR) to protect programs against common attacks. NetBSD 8.0 also includes USB3 support and the ability to work with ZFS storage volumes.

  • Early impressions

Since I had set up NetBSD with a Full install and enabled xdm during the setup process, the operating system booted to a graphical login screen. From here we can sign into our account. The login screen does not provide options to shut down or restart the computer. Logging into our account brings up the twm window manager and provides a virtual terminal, courtesy of xterm. There is a panel that provides a method for logging out of the window manager. The twm environment is sparse, fast and devoid of distractions.

  • Software management

NetBSD ships with a fairly standard collection of command line tools and manual pages, but otherwise it is a fairly minimal platform. If we want to run network services, have access to a web browser, or use a word processor we are going to need to install more software. There are two main approaches to installing new packages. The first, and easier approach, is to use the pkgin package manager. The pkgin utility works much the same way APT or DNF work in the Linux world, or as pkg works on FreeBSD. We can search for software by name, install or remove items. I found pkgin worked well, though its output can be terse. My only complaint with pkgin is that it does not handle “close enough” package names. For example, if I tried to run “pkgin install vlc” or “pkgin install firefox” I would quickly be told these items did not exist. But a more forgiving package manager will realize items like vlc2 or firefox45 are available and offer to install those.
The pkgin tool installs new programs in the /usr/pkg/bin directory. Depending on your configuration and shell, this location may not be in your user’s path, and it will be helpful to adjust your PATH variable accordingly.
The other common approach to acquiring new software is to use the pkgsrc framework. I have talked about using pkgsrc before and I will skip the details. Basically, we can download a collection of recipes for building popular open source software and run a command to download and install these items from their source code. Using pkgsrc basically gives us the same software as using pkgin would, but with some added flexibility on the options we use.
Once new software has been installed, it may need to be enabled and activated, particularly if it uses (or is) a background service. New items can be enabled in the /etc/rc.conf file and started or stopped using the service command. This works about the same as the service command on FreeBSD and most non-systemd Linux distributions.

  • Hardware

I found that, when logged into the twm environment, NetBSD used about 130MB of RAM. This included kernel memory and all active memory. A fresh, Full install used up 1.5GB of disk space. I generally found NetBSD ran well in both VirtualBox and on my desktop computer. The system was quick and stable. I did have trouble getting a higher screen resolution in both environments. NetBSD does not offer VirtualBox add-on modules. There are NetBSD patches for VirtualBox out there, but there is some manual work involved in getting them working. When running on my desktop computer I think the resolution issue was one of finding and dealing with the correct video driver. Screen resolution aside, NetBSD performed well and detected all my hardware.

  • Personal projects

Since NetBSD provides users with a small, core operating system without many utilities if we want to use NetBSD for something we need to have a project in mind. I had four mini projects in mind I wanted to try this week: install a desktop environment, enable file sharing for computers on the local network, test multimedia (video, audio and YouTube capabilities), and set up a ZFS volume for storage.
I began with the desktop. Specifically, I followed the same tutorial I used three years ago to try to set up the Xfce desktop. While Xfce and its supporting services installed, I was unable to get a working desktop out of the experience. I could get the Xfce window manager working, but not the entire session. This tutorial worked beautifully with NetBSD 7.0, but not with version 8.0. Undeterred, I switched gears and installed Fluxbox instead. This gave me a slightly more powerful graphical environment than what I had before with twm while maintaining performance. Fluxbox ran without any problems, though its application menu was automatically populated with many programs which were not actually installed.
Next, I tried installing a few multimedia applications to play audio and video files. Here I ran into a couple of interesting problems. I found the music players I installed would play audio files, but the audio was quite slow. It always sounded like a cassette tape dragging. When I tried to play a video, the entire graphical session would crash, taking me back to the login screen. When I installed Firefox, I found I could play YouTube videos, and the video played smoothly, but again the audio was unusually slow.
I set up two methods of sharing files on the local network: OpenSSH and FTP. NetBSD basically gives us OpenSSH for free at install time and I added an FTP server through the pkgin package manager which worked beautifully with its default configuration.
I experimented with ZFS support a little, just enough to confirm I could create and access ZFS volumes. ZFS seems to work on NetBSD just as well, and with the same basic features, as it does on FreeBSD and mainstream Linux distributions. I think this is a good feature for the portable operating system to have since it means we can stick NetBSD on nearly any networked computer and use it as a NAS.

  • Conclusions

NetBSD, like its close cousins (FreeBSD and OpenBSD) does not do a lot of hand holding or automation. It offers a foundation that will run on most CPUs and we can choose to build on that foundation. I mention this because, on its own, NetBSD does not do much. If we want to get something out of it, we need to be willing to build on its foundation - we need a project. This is important to keep in mind as I think going into NetBSD and thinking, “Oh I’ll just explore around and expand on this as I go,” will likely lead to disappointment. I recommend figuring out what you want to do before installing NetBSD and making sure the required tools are available in the operating system’s repositories.
Some of the projects I embarked on this week (using ZFS and setting up file sharing) worked well. Others, like getting multimedia support and a full-featured desktop, did not. Given more time, I’m sure I could find a suitable desktop to install (along with the required documentation to get it and its services running), or customize one based on one of the available window managers. However, any full featured desktop is going to require some manual work. Media support was not great. The right players and codecs were there, but I was not able to get audio to play smoothly.
My main complaint with NetBSD relates to my struggle to get some features working to my satisfaction: the documentation is scattered. There are four different sections of the project’s website for documentation (FAQs, The Guide, manual pages and the wiki). Whatever we are looking for is likely to be in one of those, but which one? Or, just as likely, the tutorial we want is not there, but is on a forum or blog somewhere. I found that the documentation provided was often thin, more of a quick reference to remind people how something works rather than a full explanation.
As an example, I found a couple of documents relating to setting up a firewall. One dealt with networking NetBSD on a LAN, another explored IPv6 support, but neither gave an overview on syntax or a basic guide to blocking all but one or two ports. It seemed like that information should already be known, or picked up elsewhere.
Newcomers are likely to be a bit confused by software management guides for the same reason. Some pages refer to using a tool called pkg_add, others use pkgsrc and its make utility, others mention pkgin. Ultimately, these tools each give approximately the same result, but work differently and yet are mentioned almost interchangeably. I have used NetBSD before a few times and could stumble through these guides, but new users are likely to come away confused.
One quirk of NetBSD, which may be a security feature or an inconvenience, depending on one’s point of view, is super user programs are not included in regular users’ paths. This means we need to change our path if we want to be able to run programs typically used by root. For example, shutdown and mount are not in regular users’ paths by default. This made checking some things tricky for me.
Ultimately though, NetBSD is not famous for its convenience or features so much as its flexibility. The operating system will run on virtually any processor and should work almost identically across multiple platforms. That gives NetBSD users a good deal of consistency across a range of hardware and the chance to experiment with a member of the Unix family on hardware that might not be compatible with Linux or the other BSDs.

###Showing a Gigabit OpenBSD Firewall Some Monitoring Love

I have a pretty long history of running my home servers or firewalls on “exotic” hardware. At first, it was Sun Microsystem hardware, then it moved to the excellent Soekris line, with some cool single board computers thrown in the mix. Recently I’ve been running OpenBSD Octeon on the Ubiquiti Edge Router Lite, an amazing little piece of kit at an amazing price point.

  • Upgrade Time!

This setup has served me for some time and I’ve been extremely happy with it. But, in the #firstworldproblems category, I recently upgraded the household to the amazing Gigabit fibre offering from Sonic. A great problem to have, but also too much of a problem for the little Edge Router Lite (ERL).
The way the OpenBSD PF firewall works, it’s only able to process packets on a single core. Not a problem for the dual-core 500 MHz ERL when you’re pushing under ~200 Mbps, but more of a problem when you’re trying to push 1000 Mbps.
I needed something that was faster on a per core basis but still satisfied my usual firewall requirements. Loosely:

  • small form factor
  • fan-less
  • multiple Intel Ethernet ports (good driver support)
  • low power consumption
  • not your regular off-the-shelf kit
  • relatively inexpensive

After evaluating a LOT of different options I settled on the Protectli Vault FW2B. With the specs required for the firewall (2 GB RAM and 8 GB drive) it comes in at a mere $239 USD! Installation of OpenBSD 6.4 was pretty straight forward, with the only problem I had was Etcher did not want to recognize the ‘.fs’ extension on the install image as bootable image. I quickly fixed this with good old Unix dd(1) on the Mac. Everything else was incredibly smooth.
After loading the same rulesets on my new install, the results were fantastic!

  • Monitoring

Now that the machine was up and running (and fast!), I wanted to know what it was doing. Over the years, I’ve always relied on the venerable pfstat software to give me an overview of my traffic, blocked packets, etc. It looks like this:
As you can see it’s based on RRDtool, which was simply incredible in its time. Having worked on monitoring almost continuously for almost the past decade, I wanted to see if we could re-implement the same functionality using more modern tools as RRDtool and pfstat definitely have their limitations. This might be an opportunity to learn some new things as well.
I came across pf-graphite which seemed to be a great start! He had everything I needed and I added a few more stats from the detailed interface statistics and the ability for the code to exit for running from cron(8), which is a bit more OpenBSD style. I added code for sending to some SaaS metrics platforms but ultimately stuck with straight Graphite. One important thing to note was to use the Graphite pickle port (2004) instead of the default plaintext port for submission. Also you will need to set a loginterface in your ‘pf.conf’.
A bit of tweaking with Graphite and Grafana, and I had a pretty darn good recreation of my original PF stats dashboard!
As you can see it’s based on RRDtool, which was simply incredible in its time. Having worked on monitoring almost continuously for almost the past decade, I wanted to see if we could re-implement the same functionality using more modern tools as RRDtool and pfstat definitely have their limitations. This might be an opportunity to learn some new things as well.
I came across pf-graphite which seemed to be a great start! He had everything I needed and I added a few more stats from the detailed interface statistics and the ability for the code to exit for running from cron(8), which is a bit more OpenBSD style. I added code for sending to some SaaS metrics platforms but ultimately stuck with straight Graphite. One important thing to note was to use the Graphite pickle port (2004) instead of the default plaintext port for submission. Also you will need to set a loginterface in your ‘pf.conf’.
A bit of tweaking with Graphite and Grafana, and I had a pretty darn good recreation of my original PF stats dashboard!

###The Source History of Cat

I once had a debate with members of my extended family about whether a computer science degree is a degree worth pursuing. I was in college at the time and trying to decide whether I should major in computer science. My aunt and a cousin of mine believed that I shouldn’t. They conceded that knowing how to program is of course a useful and lucrative thing, but they argued that the field of computer science advances so quickly that everything I learned would almost immediately be outdated. Better to pick up programming on the side and instead major in a field like economics or physics where the basic principles would be applicable throughout my lifetime.
I knew that my aunt and cousin were wrong and decided to major in computer science. (Sorry, aunt and cousin!) It is easy to see why the average person might believe that a field like computer science, or a profession like software engineering, completely reinvents itself every few years. We had personal computers, then the web, then phones, then machine learning… technology is always changing, so surely all the underlying principles and techniques change too. Of course, the amazing thing is how little actually changes. Most people, I’m sure, would be stunned to know just how old some of the important software on their computer really is. I’m not talking about flashy application software, admittedly—my copy of Firefox, the program I probably use the most on my computer, is not even two weeks old. But, if you pull up the manual page for something like grep, you will see that it has not been updated since 2010 (at least on MacOS). And the original version of grep was written in 1974, which in the computing world was back when dinosaurs roamed Silicon Valley. People (and programs) still depend on grep every day.
My aunt and cousin thought of computer technology as a series of increasingly elaborate sand castles supplanting one another after each high tide clears the beach. The reality, at least in many areas, is that we steadily accumulate programs that have solved problems. We might have to occasionally modify these programs to avoid software rot, but otherwise they can be left alone. grep is a simple program that solves a still-relevant problem, so it survives. Most application programming is done at a very high level, atop a pyramid of much older code solving much older problems. The ideas and concepts of 30 or 40 years ago, far from being obsolete today, have in many cases been embodied in software that you can still find installed on your laptop.
I thought it would be interesting to take a look at one such old program and see how much it had changed since it was first written. cat is maybe the simplest of all the Unix utilities, so I’m going to use it as my example. Ken Thompson wrote the original implementation of cat in 1969. If I were to tell somebody that I have a program on my computer from 1969, would that be accurate? How much has cat really evolved over the decades? How old is the software on our computers?
Thanks to repositories like this one, we can see exactly how cat has evolved since 1969. I’m going to focus on implementations of cat that are ancestors of the implementation I have on my Macbook. You will see, as we trace cat from the first versions of Unix down to the cat in MacOS today, that the program has been rewritten more times than you might expect—but it ultimately works more or less the same way it did fifty years ago.

  • Research Unix

Ken Thompson and Dennis Ritchie began writing Unix on a PDP 7. This was in 1969, before C, so all of the early Unix software was written in PDP 7 assembly. The exact flavor of assembly they used was unique to Unix, since Ken Thompson wrote his own assembler that added some features on top of the assembler provided by DEC, the PDP 7’s manufacturer. Thompson’s changes are all documented in the original Unix Programmer’s Manual under the entry for as, the assembler.
The first implementation of cat is thus in PDP 7 assembly. I’ve added comments that try to explain what each instruction is doing, but the program is still difficult to follow unless you understand some of the extensions Thompson made while writing his assembler. There are two important ones. First, the ; character can be used to separate multiple statements on the same line. It appears that this was used most often to put system call arguments on the same line as the sys instruction. Second, Thompson added support for “temporary labels” using the digits 0 through 9. These are labels that can be reused throughout a program, thus being, according to the Unix Programmer’s Manual, “less taxing both on the imagination of the programmer and on the symbol space of the assembler.” From any given instruction, you can refer to the next or most recent temporary label n using nf and nb respectively. For example, if you have some code in a block labeled 1:, you can jump back to that block from further down by using the instruction jmp 1b. (But you cannot jump forward to that block from above without using jmp 1f instead.)
The most interesting thing about this first version of cat is that it contains two names we should recognize. There is a block of instructions labeled getc and a block of instructions labeled putc, demonstrating that these names are older than the C standard library. The first version of cat actually contained implementations of both functions. The implementations buffered input so that reads and writes were not done a character at a time.
The first version of cat did not last long. Ken Thompson and Dennis Ritchie were able to persuade Bell Labs to buy them a PDP 11 so that they could continue to expand and improve Unix. The PDP 11 had a different instruction set, so cat had to be rewritten. I’ve marked up this second version of cat with comments as well. It uses new assembler mnemonics for the new instruction set and takes advantage of the PDP 11’s various addressing modes. (If you are confused by the parentheses and dollar signs in the source code, those are used to indicate different addressing modes.) But it also leverages the ; character and temporary labels just like the first version of cat, meaning that these features must have been retained when as was adapted for the PDP 11.
The second version of cat is significantly simpler than the first. It is also more “Unix-y” in that it doesn’t just expect a list of filename arguments—it will, when given no arguments, read from stdin, which is what cat still does today. You can also give this version of cat an argument of - to indicate that it should read from stdin.
In 1973, in preparation for the release of the Fourth Edition of Unix, much of Unix was rewritten in C. But cat does not seem to have been rewritten in C until a while after that. The first C implementation of cat only shows up in the Seventh Edition of Unix. This implementation is really fun to look through because it is so simple. Of all the implementations to follow, this one most resembles the idealized cat used as a pedagogic demonstration in K&R C. The heart of the program is the classic two-liner:

while ((c = getc(fi)) != EOF)
putchar(c);

There is of course quite a bit more code than that, but the extra code is mostly there to ensure that you aren’t reading and writing to the same file. The other interesting thing to note is that this implementation of cat only recognized one flag, -u. The -u flag could be used to avoid buffering input and output, which cat would otherwise do in blocks of 512 bytes.

  • BSD

After the Seventh Edition, Unix spawned all sorts of derivatives and offshoots. MacOS is built on top of Darwin, which in turn is derived from the Berkeley Software Distribution (BSD), so BSD is the Unix offshoot we are most interested in. BSD was originally just a collection of useful programs and add-ons for Unix, but it eventually became a complete operating system. BSD seems to have relied on the original cat implementation up until the fourth BSD release, known as 4BSD, when support was added for a whole slew of new flags. The 4BSD implementation of cat is clearly derived from the original implementation, though it adds a new function to implement the behavior triggered by the new flags. The naming conventions already used in the file were adhered to—the fflg variable, used to mark whether input was being read from stdin or a file, was joined by nflg, bflg, vflg, sflg, eflg, and tflg, all there to record whether or not each new flag was supplied in the invocation of the program. These were the last command-line flags added to cat; the man page for cat today lists these flags and no others, at least on Mac OS. 4BSD was released in 1980, so this set of flags is 38 years old.
cat would be entirely rewritten a final time for BSD Net/2, which was, among other things, an attempt to avoid licensing issues by replacing all AT&T Unix-derived code with new code. BSD Net/2 was released in 1991. This final rewrite of cat was done by Kevin Fall, who graduated from Berkeley in 1988 and spent the next year working as a staff member at the Computer Systems Research Group (CSRG). Fall told me that a list of Unix utilities still implemented using AT&T code was put up on a wall at CSRG and staff were told to pick the utilities they wanted to reimplement. Fall picked cat and mknod. The cat implementation bundled with MacOS today is built from a source file that still bears his name at the very top. His version of cat, even though it is a relatively trivial program, is today used by millions.
Fall’s original implementation of cat is much longer than anything we have seen so far. Other than support for a -? help flag, it adds nothing in the way of new functionality. Conceptually, it is very similar to the 4BSD implementation. It is only longer because Fall separates the implementation into a “raw” mode and a “cooked” mode. The “raw” mode is cat classic; it prints a file character for character. The “cooked” mode is cat with all the 4BSD command-line options. The distinction makes sense but it also pads out the implementation so that it seems more complex at first glance than it actually is. There is also a fancy error handling function at the end of the file that further adds to its length.

  • MacOS

The very first release of Mac OS X thus includes an implementation of cat pulled from the NetBSD project. So the first Mac OS X implementation of cat is Kevin Fall’s cat. The only thing that had changed over the intervening decade was that Fall’s error-handling function err() was removed and the err() function made available by err.h was used in its place. err.h is a BSD extension to the C standard library.
The NetBSD implementation of cat was later swapped out for FreeBSD’s implementation of cat. According to Wikipedia, Apple began using FreeBSD instead of NetBSD in Mac OS X 10.3 (Panther). But the Mac OS X implementation of cat, according to Apple’s own open source releases, was not replaced until Mac OS X 10.5 (Leopard) was released in 2007. The FreeBSD implementation that Apple swapped in for the Leopard release is the same implementation on Apple computers today. As of 2018, the implementation has not been updated or changed at all since 2007.
So the Mac OS cat is old. As it happens, it is actually two years older than its 2007 appearance in MacOS X would suggest. This 2005 change, which is visible in FreeBSD’s Github mirror, was the last change made to FreeBSD’s cat before Apple pulled it into Mac OS X. So the Mac OS X cat implementation, which has not been kept in sync with FreeBSD’s cat implementation, is officially 13 years old. There’s a larger debate to be had about how much software can change before it really counts as the same software; in this case, the source file has not changed at all since 2005.
The cat implementation used by Mac OS today is not that different from the implementation that Fall wrote for the 1991 BSD Net/2 release. The biggest difference is that a whole new function was added to provide Unix domain socket support. At some point, a FreeBSD developer also seems to have decided that Fall’s rawargs() function and cookargs() should be combined into a single function called scanfiles(). Otherwise, the heart of the program is still Fall’s code.
I asked Fall how he felt about having written the cat implementation now used by millions of Apple users, either directly or indirectly through some program that relies on cat being present. Fall, who is now a consultant and a co-author of the most recent editions of TCP/IP Illustrated, says that he is surprised when people get such a thrill out of learning about his work on cat. Fall has had a long career in computing and has worked on many high-profile projects, but it seems that many people still get most excited about the six months of work he put into rewriting cat in 1989.

  • The Hundred-Year-Old Program

In the grand scheme of things, computers are not an old invention. We’re used to hundred-year-old photographs or even hundred-year-old camera footage. But computer programs are in a different category—they’re high-tech and new. At least, they are now. As the computing industry matures, will we someday find ourselves using programs that approach the hundred-year-old mark?
Computer hardware will presumably change enough that we won’t be able to take an executable compiled today and run it on hardware a century from now. Perhaps advances in programming language design will also mean that nobody will understand C in the future and cat will have long since been rewritten in another language. (Though C has already been around for fifty years, and it doesn’t look like it is about to be replaced any time soon.) But barring all that, why not just keep using the cat we have forever?
I think the history of cat shows that some ideas in computer science are in fact very durable. Indeed, with cat, both the idea and the program itself are old. It may not be accurate to say that the cat on my computer is from 1969. But I could make a case for saying that the cat on my computer is from 1989, when Fall wrote his implementation of cat. Lots of other software is just as ancient. So maybe we shouldn’t think of computer science and software development primarily as fields that disrupt the status quo and invent new things. Our computer systems are built out of historical artifacts. At some point, we may all spend more time trying to understand and maintain those historical artifacts than we spend writing new code.

##News Roundup
Trivial Bug in X.Org Gives Root Permission on Linux and BSD Systems

A vulnerability that is trivial to exploit allows privilege escalation to root level on Linux and BSD distributions using X.Org server, the open source implementation of the X Window System that offers the graphical environment.
The flaw is now identified as CVE-2018-14665 (credited to security researcher Narendra Shinde). It has been present in xorg-server for two years, since version 1.19.0 and is exploitable by a limited user as long as the X server runs with elevated permissions.

  • Privilege escalation and arbitrary file overwrite

An advisory on Thursday describes the problem as an “incorrect command-line parameter validation” that also allows an attacker to overwrite arbitrary files.
Privilege escalation can be accomplished via the -modulepath argument by setting an insecure path to modules loaded by the X.org server. Arbitrary file overwrite is possible through the -logfile argument, because of improper verification when parsing the option.

  • Bug could have been avoided in OpenBSD 6.4

OpenBSD, the free and open-source operating system with a strong focus on security, uses xorg. On October 18, the project released version 6.4 of the OS, affected by CVE-2018-14665. This could have been avoided, though.
Theo de Raadt, founder and leader of the OpenBSD project, says that X maintainer knew about the problem since at least October 11. For some reason, the OpenBSD developers received the message one hour before the public announcement this Thursday, a week after their new OS release.
“As yet we don’t have answers about why our X maintainer (on the X security team) and his team provided information to other projects (some who don’t even ship with this new X server) but chose to not give us a heads-up which could have saved all the new 6.4 users a lot of grief,” Raadt says.
Had OpenBSD developers known about the bug before the release, they could have taken steps to mitigate the problem or delay the launch for a week or two.
To remedy the problem, the OpenBSD project provides a source code patch, which requires compiling and rebuilding the X server.
As a temporary solution, users can disable the Xorg binary by running the following command:

chmod u-s /usr/X11R6/bin/Xorg

  • Trivial exploitation

CVE-2018-14665 does not help compromise systems, but it is useful in the following stages of an attack.
Leveraging it after gaining access to a vulnerable machine is fairly easy. Matthew Hickey, co-founder, and head of Hacker House security outfit created and published an exploit, saying that it can be triggered from a remote SSH session.
Three hours after the public announcement of the security gap, Daemon Security CEO Michael Shirk replied with one line that overwrote shadow files on the system. Hickey did one better and fit the entire local privilege escalation exploit in one line.
Apart from OpenBSD, other operating systems affected by the bug include Debian and Ubuntu, Fedora and its downstream distro Red Hat Enterprise Linux along with its community-supported counterpart CentOS.

###OpenBSD on the Desktop: some thoughts

I’ve been using OpenBSD on my ThinkPad X230 for some weeks now, and the experience has been peculiar in some ways.
The OS itself in my opinion is not ready for widespread desktop usage, and the development team is not trying to push it in the throat of anybody who wants a Windows or macOS alternative.
You need to understand a little bit of how *NIX systems work, because you’ll use CLI more than UI.
That’s not necessarily bad, and I’m sure I learned a trick or two that could translate easily to Linux or macOS.
Their development process is purely based on developers that love to contribute and hack around, just because it’s fun.
Even the mailing list is a cool place to hang on!
Code correctness and security are a must, nothing gets committed if it doesn’t get reviewed thoroughly first - nowadays the first two properties should be enforced in every major operating system.
I like the idea of a platform that continually evolves.
pledge(2) and unveil(2) are the proof that with a little effort, you can secure existing software better than ever.
I like the “sensible defaults” approach, having an OS ready to be used - UI included if you selected it during the setup process - is great.
Just install a browser and you’re ready to go.
Manual pages on OpenBSD are real manuals, not an extension of the “–help” command found in most CLI softwares.
They help you understand inner workings of the operating system, no internet connection needed.
There are some trade-offs, too.
Performance is not first-class, mostly because of all the security mitigations and checks done at runtime3.
I write Go code in neovim, and sometimes you can feel a slight slowdown when you’re compiling and editing multiple files at the same time, but usually I can’t notice any meaningful difference.
Browsers are a different matter though, you can definitely feel something differs from the experience you can have on mainstream operating systems.
But again, trade-offs.
To use OpenBSD on the desktop you must be ready to sacrifice some of the goodies of mainstream OSes, but if you’re searching for a zen place to do your computing stuff, it’s the best you can get right now.

###Review: NomadBSD 1.1

One of the most recent additions to the DistroWatch database is NomadBSD. According to the NomadBSD website: “NomadBSD is a 64-bit live system for USB flash drives, based on FreeBSD. Together with automatic hardware detection and setup, it is configured to be used as a desktop system that works out of the box, but can also be used for data recovery.”
The latest release of NomadBSD (or simply “Nomad”, as I will refer to the project in this review) is version 1.1. It is based on FreeBSD 11.2 and is offered in two builds, one for generic personal computers and one for Macbooks. The release announcement mentions version 1.1 offers improved video driver support for Intel and AMD cards. The operating system ships with Octopkg for graphical package management and the system should automatically detect, and work with, VirtualBox environments.
Nomad 1.1 is available as a 2GB download, which we then decompress to produce a 4GB file which can be written to a USB thumb drive. There is no optical media build of Nomad as it is designed to be run entirely from the USB drive, and write data persistently to the drive, rather than simply being installed from the USB media.

  • Initial setup

Booting from the USB drive brings up a series of text-based menus which ask us to configure key parts of the operating system. We are asked to select our time zone, keyboard layout, keyboard model, keyboard mapping and our preferred language. While we can select options from a list, the options tend to be short and cryptic. Rather than “English (US)”, for example, we might be given “enUS”. We are also asked to create a password for the root user account and another one for a regular user which is called “nomad”. We can then select which shell nomad will use. The default is zsh, but there are plenty of other options, including csh and bash. We have the option of encrypting our user’s home directory.
I feel it is important to point out that these settings, and nomad’s home directory, are stored on the USB drive. The options and settings we select will not be saved to our local hard drive and our configuration choices will not affect other operating systems already installed on our computer. At the end, the configuration wizard asks if we want to run the BSDstats service. This option is not explained at all, but it contacts BSDstats to provide some basic statistics on BSD users.
The system then takes a few minutes to apply its changes to the USB drive and automatically reboots the computer. While running the initial setup wizard, I had nearly identical experiences when running Nomad on a physical computer and running the operating system in a VirtualBox virtual machine. However, after the initial setup process was over, I had quite different experiences depending on the environment so I want to divide my experiences into two different sections.

  • Physical desktop computer

At first, Nomad failed to boot on my desktop computer. From the operating system’s boot loader, I enabled Safe Mode which allowed Nomad to boot. At that point, Nomad was able to start up, but would only display a text console. The desktop environment failed to start when running in Safe Mode.
Networking was also disabled by default and I had to enable a network interface and DHCP address assignment to connect to the Internet. Instructions for enabling networking can be found in FreeBSD’s Handbook. Once we are on-line we can use the pkg command line package manager to install and update software. Had the desktop environment worked then the Octopkg graphical package manager would also be available to make browsing and installing software a point-n-click experience.
Had I been able to run the desktop for prolonged amounts of time I could have made use of such pre-installed items as the Firefox web browser, the VLC media player, LibreOffice and Thunderbird. Nomad offers a fairly small collection of desktop applications, but what is there is mostly popular, capable software.
When running the operating system I noted that, with one user logged in, Nomad only runs 15 processes with the default configuration. These processes require less than 100MB of RAM, and the whole system fits comfortably on a 4GB USB drive.

  • Conclusions

Ultimately using Nomad was not a practical option for me. The operating system did not work well with my hardware, or the virtual environment. In the virtual machine, Nomad crashed consistently after just a few minutes of uptime. On the desktop computer, I could not get a desktop environment to run. The command line tools worked well, and the system performed tasks very quickly, but a command line only environment is not well suited to my workflow.
I like the idea of what NomadBSD is offering. There are not many live desktop flavours of FreeBSD, apart from GhostBSD. It was nice to see developers trying to make a FreeBSD-based, plug-and-go operating system that would offer a desktop and persistent storage. I suspect the system would work and perform its stated functions on different hardware, but in my case my experiment was necessarily short lived.

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 22 Nov 2018 23:00:00 -0800
Episode 272: Detain the bhyve | BSD Now 272
Byproducts of reading OpenBSD’s netcat code, learnings from porting your own projects to FreeBSD, OpenBSD’s unveil(), NetBSD’s Virtual Machine Monitor, what 'dependency' means in Unix init systems, jailing bhyve, and more.

##Headlines
###The byproducts of reading OpenBSD netcat code

When I took part in a training last year, I heard about netcat for the first time. During that class, the tutor showed some hacks and tricks of using netcat which appealed to me and motivated me to learn the guts of it. Fortunately, in the past 2 months, I was not so busy that I can spend my spare time to dive into OpenBSD‘s netcat source code, and got abundant byproducts during this process.
(1) Brush up socket programming. I wrote my first network application more than 10 years ago, and always think the socket APIs are marvelous. Just ~10 functions (socket, bind, listen, accept…) with some IO multiplexing buddies (select, poll, epoll…) connect the whole world, wonderful! From that time, I developed a habit that is when touching a new programming language, network programming is an essential exercise. Even though I don’t write socket related code now, reading netcat socket code indeed refresh my knowledge and teach me new stuff.
(2) Write a tutorial about netcat. I am mediocre programmer and will forget things when I don’t use it for a long time. So I just take notes of what I think is useful. IMHO, this “tutorial” doesn’t really mean teach others something, but just a journal which I can refer when I need in the future.
(3) Submit patches to netcat. During reading code, I also found bugs and some enhancements. Though trivial contributions to OpenBSD, I am still happy and enjoy it.
(4) Implement a C++ encapsulation of libtls. OpenBSD‘s netcat supports tls/ssl connection, but it needs you take full care of resource management (memory, socket, etc), otherwise a small mistake can lead to resource leak which is fatal for long-live applications (In fact, the two bugs I reported to OpenBSD are all related resource leak). Therefore I develop a simple C++ library which wraps the libtls and hope it can free developer from this troublesome problem and put more energy in application logic part.
Long story to short, reading classical source code is a rewarding process, and you can consider to try it yourself.

###What I learned from porting my projects to FreeBSD

  • Introduction

I set up a local FreeBSD VirtualBox VM to test something, and it seems to work very well. Due to the novelty factor, I decided to get my software projects to build and pass the tests there.

##News Roundup
###OpenBSD’s unveil()

One of the key aspects of hardening the user-space side of an operating system is to provide mechanisms for restricting which parts of the filesystem hierarchy a given process can access. Linux has a number of mechanisms of varying capability and complexity for this purpose, but other kernels have taken a different approach. Over the last few months, OpenBSD has inaugurated a new system call named unveil() for this type of hardening that differs significantly from the mechanisms found in Linux.
The value of restricting access to the filesystem, from a security point of view, is fairly obvious. A compromised process cannot exfiltrate data that it cannot read, and it cannot corrupt files that it cannot write. Preventing unwanted access is, of course, the purpose of the permissions bits attached to every file, but permissions fall short in an important way: just because a particular user has access to a given file does not necessarily imply that every program run by that user should also have access to that file. There is no reason why your PDF viewer should be able to read your SSH keys, for example. Relying on just the permission bits makes it easy for a compromised process to access files that have nothing to do with that process’s actual job.
In a Linux system, there are many ways of trying to restrict that access; that is one of the purposes behind the Linux security module (LSM) architecture, for example. The SELinux LSM uses a complex matrix of labels and roles to make access-control decisions. The AppArmor LSM, instead, uses a relatively simple table of permissible pathnames associated with each application; that approach was highly controversial when AppArmor was first merged, and is still looked down upon by some security developers. Mount namespaces can be used to create a special view of the filesystem hierarchy for a set of processes, rendering much of that hierarchy invisible and, thus, inaccessible. The seccomp mechanism can be used to make decisions on attempts by a process to access files, but that approach is complex and error-prone. Yet another approach can be seen in the Qubes OS distribution, which runs applications in virtual machines to strictly control what they can access.
Compared to many of the options found in Linux, unveil() is an exercise in simplicity. This system call, introduced in July, has this prototype:

int unveil(const char *path, const char *permissions);

A process that has never called unveil() has full access to the filesystem hierarchy, modulo the usual file permissions and any restrictions that may have been applied by calling pledge(). Calling unveil() for the first time will “drop a veil” across the entire filesystem, rendering the whole thing invisible to the process, with one exception: the file or directory hierarchy starting at path will be accessible with the given permissions. The permissions string can contain any of “r” for read access, “w” for write, “x” for execute, and “c” for the ability to create or remove the path.
Subsequent calls to unveil() will make other parts of the filesystem hierarchy accessible; the unveil() system call itself still has access to the entire hierarchy, so there is no problem with unveiling distinct subtrees that are, until the call is made, invisible to the process. If one unveil() call applies to a subtree of a hierarchy unveiled by another call, the permissions associated with the more specific call apply.
Calling unveil() with both arguments as null will block any further calls, setting the current view of the filesystem in stone. Calls to unveil() can also be blocked using pledge(). Either way, once the view of the filesystem has been set up appropriately, it is possible to lock it so that the process cannot expand its access in the future should it be taken over and turn hostile.
unveil() thus looks a bit like AppArmor, in that it is a path-based mechanism for restricting access to files. In either case, one must first study the program in question to gain a solid understanding of which files it needs to access before closing things down, or the program is likely to break. One significant difference (beyond the other sorts of behavior that AppArmor can control) is that AppArmor’s permissions are stored in an external policy file, while unveil() calls are made by the application itself. That approach keeps the access rules tightly tied to the application and easy for the developers to modify, but it also makes it harder for system administrators to change them without having to rebuild the application from source.
One can certainly aim a number of criticisms at unveil() — all of the complaints that have been leveled at path-based access control and more. But the simplicity of unveil() brings a certain kind of utility, as can be seen in the large number of OpenBSD applications that are being modified to use it. OpenBSD is gaining a base level of protection against unintended program behavior; while it is arguably possible to protect a Linux system to a much greater extent, the complexity of the mechanisms involved keeps that from happening in a lot of real-world deployments. There is a certain kind of virtue to simplicity in security mechanisms.

###NetBSD Virtual Machine Monitor (NVVM)

  • NetBSD Virtual Machine Monitor

The NVMM driver provides hardware-accelerated virtualization support on NetBSD. It is made of an ~MI frontend, to which MD backends can be plugged. A virtualization API is provided in libnvmm, that allows to easily create and manage virtual machines via NVMM. Two additional components are shipped as demonstrators, toyvirt and smallkern: the former is a toy virtualizer, that executes in a VM the 64bit ELF binary given as argument, the latter is an example of such binary.

  • Download

The source code of NVMM, plus the associated tools, can be downloaded here.

  • Technical details

NVMM can support up to 128 virtual machines, each having a maximum of 256 VCPUs and 4GB of RAM.
Each virtual machine is granted access to most of the CPU registers: the GPRs (obviously), the Segment Registers, the Control Registers, the Debug Registers, the FPU (x87 and SSE), and several MSRs.
Events can be injected in the virtual machines, to emulate device interrupts. A delay mechanism is used, and allows VMM software to schedule the interrupt right when the VCPU can receive it. NMIs can be injected as well, and use a similar mechanism.
The host must always be x8664, but the guest has no constraint on the mode, so it can be x8632, PAE, real mode, and so on.
The TSC of each VCPU is always re-based on the host CPU it is executing on, and is therefore guaranteed to increase regardless of the host CPU. However, it may not increase monotonically, because it is not possible to fully hide the host effects on the guest during #VMEXITs.
When there are more VCPUs than the host TLB can deal with, NVMM uses a shared ASID, and flushes the shared-ASID VCPUs on each VM switch.
The different intercepts are configured in such a way that they cover everything that needs to be emulated. In particular, the LAPIC can be emulated by VMM software, by intercepting reads/writes to the LAPIC page in memory, and monitoring changes to CR8 in the exit state.

###What ‘dependency’ means in Unix init systems is underspecified (utoronto.ca)

I was reading Davin McCall’s On the vagaries of init systems (via) when I ran across the following, about the relationship between various daemons (services, etc):
I do not see any compelling reason for having ordering relationships without actual dependency, as both Nosh and Systemd provide for. In comparison, Dinit’s dependencies also imply an ordering, which obviates the need to list a dependency twice in the service description.
Well, this may be an easy one but it depends on what an init system means by ‘dependency’. Let’s consider ®syslog and the SSH daemon. I want the syslog daemon to be started before the SSH daemon is started, so that the SSH daemon can log things to it from the beginning. However, I very much do not want the SSH daemon to not be started (or to be shut down) if the syslog daemon fails to start or later fails. If syslog fails, I still want the SSH daemon to be there so that I can perhaps SSH in to the machine and fix the problem.
This is generally true of almost all daemons; I want them to start after syslog, so that they can syslog things, but I almost never want them to not be running if syslog failed. (And if for some reason syslog is not configured to start, I want enabling and starting, say, SSH, to also enable and start the syslog daemon.)
In general, there are three different relationships between services that I tend to encounter:

  • a hard requirement, where service B is useless or dangerous without service A. For instance, many NFS v2 and NFS v3 daemons basically don’t function without the RPC portmapper alive and active. On any number of systems, firewall rules being in place are a hard requirement to start most network services; you would rather your network services not start at all than that they start without your defenses in place.

  • a want, where service B wants service A to be running before B starts up, and service A should be started even if it wouldn’t otherwise be, but the failure of A still leaves B functional. Many daemons want the syslog daemon to be started before they start but will run without it, and often you want them to do so so that at least some of the system works even if there is, say, a corrupt syslog configuration file that causes the daemon to error out on start. (But some environments want to hard-fail if they can’t collect security related logging information, so they might make rsyslogd a requirement instead of a want.)

  • an ordering, where if service A is going to be started, B wants to start after it (or before it), but B isn’t otherwise calling for A to be started. We have some of these in our systems, where we need NFS mounts done before cron starts and runs people’s @reboot jobs but neither cron nor NFS mounts exactly or explicitly want each other. (The system as a whole wants both, but that’s a different thing.)

Given these different relationships and the implications for what the init system should do in different situations, talking about ‘dependency’ in it systems is kind of underspecified. What sort of dependency? What happens if one service doesn’t start or fails later?
My impression is that generally people pick a want relationship as the default meaning for init system ‘dependency’. Usually this is fine; most services aren’t actively dangerous if one of their declared dependencies fails to start, and it’s generally harmless on any particular system to force a want instead of an ordering relationship because you’re going to be starting everything anyway.

  • (In my example, you might as well say that cron on the systems in question wants NFS mounts. There is no difference in practice; we already always want to do NFS mounts and start cron.)

###Jailing The bhyve Hypervisor

As FreeBSD nears the final 12.0-RELEASE release engineering cycles, I’d like to take a moment to document a cool new feature coming in 12: jailed bhyve.
You may notice that I use HardenedBSD instead of FreeBSD in this article. There is no functional difference in bhyve on HardenedBSD versus bhyve on FreeBSD. The only difference between HardenedBSD and FreeBSD is the aditional security offered by HardenedBSD.
The steps I outline here work for both FreeBSD and HardenedBSD. These are the bare minimum steps, no extra work needed for either FreeBSD or HardenedBSD.

  • A Gentle History Lesson

At work in my spare time, I’m helping develop a malware lab. Due to the nature of the beast, we would like to use bhyve on HardenedBSD. Starting with HardenedBSD 12, non-Cross-DSO CFI, SafeStack, Capsicum, ASLR, and strict W^X are all applied to bhyve, making it an extremely hardened hypervisor.
So, the work to support jailed bhyve is sponsored by both HardenedBSD and my employer. We’ve also jointly worked on other bhyve hardening features, like protecting the VM’s address space using guard pages (mmap(…, MAPGUARD, …)). Further work is being done in a project called “malhyve.” Only those modifications to bhyve/malhyve that make sense to upstream will be upstreamed.

  • Initial Setup

We will not go through the process of creating the jail’s filesystem. That process is documented in the FreeBSD Handbook. For UEFI guests, you will need to install the uefi-edk2-bhyve package inside the jail.
I network these jails with traditional jail networking. I have tested vnet jails with this setup, and that works fine, too. However, there is no real need to hook the jail up to any network so long as bhyve can access the tap device. In some cases, the VM might not need networking, in which case you can use a network-less VM in a network-less jail.
By default, access to the kernel side of bhyve is disabled within jails. We need to set allow.vmm in our jail.conf entry for the bhyve jail.

  • We will use the following in our jail, so we will need to set up devfs(8) rules for them:

  • A ZFS volume

  • A null-modem device (nmdm(4))

  • UEFI GOP (no devfs rule, but IP assigned to the jail)

  • A tap device

  • Conclusion

The bhyve hypervisor works great within a jail. When combined with HardenedBSD, bhyve is extremely hardened:

  • PaX ASLR is fully applied due to compilation as a Position-Independent Executable (HardenedBSD enhancement)
  • PaX NOEXEC is fully applied (strict W^X) (HardenedBSD enhancement)
  • Non-Cross-DSO CFI is fully applied (HardenedBSD enhancement)
  • Full RELRO (RELRO + BINDNOW) is fully applied (HardenedBSD enhancement)
  • SafeStack is applied to the application (HardenedBSD enhancement)
  • Jailed (FreeBSD feature written by HardenedBSD)
  • Virtual memory protected with guard pages (FreeBSD feature written by HardenedBSD)
  • Capsicum is fully applied (FreeBSD feature)

Bad guys are going to have a hard time breaking out of the userland components of bhyve on HardenedBSD. :)

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 15 Nov 2018 10:00:00 -0800
Episode 271: Automatic Drive Tests | BSD Now 271
MidnightBSD 1.0 released, MeetBSD review, EuroBSDcon trip reports, DNS over TLS in FreeBSD 12, Upgrading OpenBSD with Ansible, how to use smartd to run tests on your drives automatically, and more.

##Headlines
MidnightBSD 1.0 now available

I’m happy to announce the availability of MidnightBSD 1.0 for amd64 and i386. Over the years, many ambitious goals were set for our 1.0 release. As it approached, it was clear we wouldn’t be able to accomplish all of them. This release is more of a natural progression rather than a groundbreaking event. It includes many updates to the base system, improvements to the package manager, an updated compiler, and tools.
Of particular note, you can now boot off of ZFS and use NVME SSDs and some AMD Radeon graphics cards support acceleration. AMD Ryzen support has greatly improved in this release. We also have added bhyve from FreeBSD.
The 1.0 release is finally available. Still building packages for i386 and plan to do an amd64 package build later in the week. The single largest issue with the release process has been the web server performance. The CPU is overloaded and has been at solid 100% for several days. The server has a core i7 7700 in it. I’m trying to figure out what to buy as an upgrade so that we don’t continue to have this issue going forward. As it’s actually blocked in multiple processes, a 6 or 8 core chip might be an improvement for the workload…

###MeetBSD Review

MeetBSD 2018 took place at the sprawling Intel Santa Clara campus. The venue itself felt more like an olive branch than a simple friendly gesture by Intel. In truth it felt like a bit of an apology. You get the subtle sense they feel bad about how the BSD’s were treated with the Meltdown and Specter flaws. In fact, you may be right to think they felt a bit sorry towards the entire open source community.

  • MeetBSD 2018

At most massive venues the parking is the first concern, not so here - in fact that was rather straightforward. No, the real challenge is navigating the buildings. Luckily I had help from navigator extraordinaire, Hadea, who located the correct building, SC12 quickly. Finding the entrance took a moment or two though. The lobby itself was converted by iXsystems efficiently into the MeetBSD expo hall, clean, efficient and roomy with registration, some seating, and an extra conference room for on-on-one sessions. On day two sponsor booths were also setup. All who showed up on day one were warmly greeted with badges, lanyards and goodies by Denise and her friendly team.
Like every great BSD event, plenty of food was made available. And as always they make it look effortless. These events showcase iXsystem’s inherent generosity toward its community; with breakfast items in the back of the main auditorium room in the morning, boxed lunches, fruit and cookies at lunch time, and snacks for the rest of the day. But just in case your still hungry, there is a pizza meetup in another Intel room after day one and two.
MeetBSD leverages it’s realistically small crowd size on day one. The morning starts off with introductions of the entire group, the mic is passed around the room.
The group is a good mix of pros in the industry (such as Juniper, Intel, Ebay, Groupon, Cisco, etc), iX staff, and a few enthusiast. Lots of people with a focus or passion for networking. And, of course, some friendly Linux bashing went down for good measure, always followed by a good natured chuckle.

  • MeetBSD Gives me The Feels

I find that I am subtly unnerved at this venue, and at lunch I saw it clearly. I have always had a strong geek radar, allowing me to navigate a new area (like Berkeley for MeetBSD of 2016, or even SCALE earlier this year in Pasadena), and in a glance I can see who is from my conference and who isn’t. This means it is easy, nearly effortless to know who to greet with a smile and a wave. These are MY people. Here at the Intel campus though it is different. The drive in alone reveals behemoth complexes all with well known tech names prominently displayed. This is Silicon Valley, and all of these people look like MY people. So much for knowing who’s from my conference. Thank goodness for those infamous BSD horns. None-the-less I am struck by how massive these tech giants are. And Intel is one of the largest of those giants, and see the physical reminders of this fact brought home the significance that they had opened their doors, wifi, and bathrooms to the BSD community.

###[EuroBSDcon 2018 Trip Reports]
https://www.freebsdfoundation.org/blog/eurobsd-2018-trip-report-joseph-mingrone/
https://www.freebsdfoundation.org/blog/eurobsd-2018-trip-report-vinicius-zavam/
https://www.freebsdfoundation.org/blog/eurobsd-2018-trip-report-emmanuel-vadot/

##News Roundup
DNS over TLS in FreeBSD 12

With the arrival of OpenSSL 1.1.1, an upgraded Unbound, and some changes to the setup and init scripts, FreeBSD 12.0, currently in beta, now supports DNS over TLS out of the box.
DNS over TLS is just what it sounds like: DNS over TCP, but wrapped in a TLS session. It encrypts your requests and the server’s replies, and optionally allows you to verify the identity of the server. The advantages are protection against eavesdropping and manipulation of your DNS traffic; the drawbacks are a slight performance degradation and potential firewall traversal issues, as it runs over a non-standard port (TCP port 853) which may be blocked on some networks. Let’s take a look at how to set it up.

  • Conclusion

We’ve seen how to set up Unbound—specifically, the local_unbound service in FreeBSD 12.0—to use DNS over TLS instead of plain UDP or TCP, using Cloudflare’s public DNS service as an example. We’ve looked at the performance impact, and at how to ensure (and verify) that Unbound validates the server certificate to prevent man-in-the-middle attacks.
The question that remains is whether it is all worth it. There is undeniably a performance hit, though this may improve with TLS 1.3. More importantly, there are currently very few DNS-over-TLS providers—only one, really, since Quad9 filter their responses—and you have to weigh the advantage of encrypting your DNS traffic against the disadvantage of sending it all to a single organization. I can’t answer that question for you, but I can tell you that the parameters are evolving quickly, and if your answer is negative today, it may not remain so for long. More providers will appear. Performance will improve with TLS 1.3 and QUIC. Within a year or two, running DNS over TLS may very well become the rule rather than the experimental exception.

###Upgrading OpenBSD with Ansible

  • My router runs OpenBSD -current

A few months ago, I needed software that had just hit the ports tree. I didn’t want to wait for the next release, so I upgraded my router to use -current. Since then, I’ve continued running -current, which means upgrading to a newer snapshot every so often. Running -current is great, but the process of updating to a newer snapshot was cumbersome. Initially, I had to plug in a serial cable and then reboot into bsd.rd, hit enter ten times, then reboot, run sysmerge and update packages.
I eventually switched to upobsd to be able to upgrade without the need for a serial connection. The process was better, but still tiresome. Usually, I would prepare the special version of bsd.rd, boot on bsd.rd, and do something like wash the dishes in the meantime. After about ten minutes, I would dry my hands and then go back to my workstation to see whether the bsd.rd part had finished so I could run sysmerge and pkgadd, and then return to the dishes while it upgraded packages.
Out of laziness, I thought: “I should automate this,” but what happened instead is that I simply didn’t upgrade that machine very often. (Yes, laziness). With my router out of commission, life is very dull, because it is my gateway to the Internet. Even services hosted at my place (like my Mastodon instance) are not reachable when the router is down because I use multiple VLANs (so I need the router to jump across VLANs).

  • Ansible Reboot Module

I recently got a new job, and one of my first tasks was auditing the Ansible roles written by my predecessors. In one role, the machine rebooted and they used the waitforconnection module to wait for it to come back up. That sounded quite hackish to me, so out of curiosity, I tried to determine whether there was a better way. I also thought I might be able to use something similar to further automate my OpenBSD upgrades, and wanted to assess the cleanliness of this method. ;-)
I learned that with the then-upcoming 2.7 Ansible release, a proper reboot module would be included. I went to the docs, which stated that for a certain parameter:
I took this to mean that there was no support for OpenBSD. I looked at the code and, indeed, there was not. However, I believed that it wouldn’t be too hard to add it. I added the missing pieces for OpenBSD, tested it on my poor Pine64 and then submitted it upstream. After a quick back and forth, the module’s author merged it into devel (having a friend working at Red Hat helped the process, merci Cyril !) A couple days later, the release engineer merged it into stable-2.7.
I proceeded to actually write the playbook, and then I hit a bug. The parameter reboottimeout was not recognized by Ansible. This feature would definitely be useful on a slow machine (such as the Pine64 and its dying SD card). Again, my fix was merged into master by the module’s author and then merged into stable-2.7. 2.7.1 will be the first release to feature these fixes, but if you use OpenBSD -current, you already have access to them. I backported the patches when I updated ansible.
Fun fact about Ansible and reboots: “The winreboot module was […] included with Ansible 2.1,” while for unix systems it wasn’t added until 2.7. :D For more details, you can read the module’s author blog article.

  • The explanations

Ansible runs my script on the remote host to fetch the sets. It creates an answer file from the template and then gives it to upobsd. Once upobsd has created the kernel, Ansible copies it in place of /bsd on the host. The router reboots and boots on /bsd, which is upobsd’s bsd.rd. The installer runs in autoupdate mode. Once it comes back from bsd.rd land, it archives the kernel and finishes by upgrading all the packages.
It also supports upgrading without fetching the sets ahead of time. For instance, I upgrade this way on my Pine64 because if I cared about speed, I wouldn’t use this weak computer with its dying SD card. For this case, I just comment out the pathsets variable and Ansible instead creates an answer file that will instruct the installer to fetch the sets from the designated mirror.
I’ve been archiving my kernels for a few years. It’s a nice way to fill up / keep a history of my upgrades. If I spot a regression, I can try a previous kernel … which may not work with the then-desynchronized userland, but that’s another story.
sysmerge already runs with rc.sysmerge in batch mode and sends the result by email. I don’t think there’s merit to running it again in the playbook. The only perk would be discovering in the terminal whether any files need to be manually merged, rather than reading exactly the same output in the email.
Initially, I used the openbsdpkg module, but it doesn’t work on -current just before a release because pkgadd automatically looks for pub/OpenBSD/${release}/packages/${arch} (which is empty). I wrote and tested this playbook while 6.4 was around the corner, so I switched to command to be able to pass the -Dsnap parameter.

  • The result

I’m very happy with the playbook! It performs the upgrade with as little intervention as possible and minimal downtime. \o/

###Using smartd to automatically run tests on your drives

Those programs can “control and monitor storage systems using the Self-Monitoring, Analysis and Reporting Technology System (SMART) built into most modern ATA/SATA, SCSI/SAS and NVMe disks. In many cases, these utilities will provide advanced warning of disk degradation and failure.” See the smartmontools website for more information.

NOTE: “Due to OS-specific issues and also depending on the different state of smartmontools development on the platforms, device support is not the same for all OS platforms.” – use the documentation for your OS.

I first started using smartd in March 2010 (according to that blog post, that’s when I still writing on both The FreeBSD Diary and this blog). Back then, and until recently, all I did was start smartd. As far as I can tell, all it did was send daily status messages via the FreeBSD periodic tools. I would set my drive devices via dailystatussmartdevices in /etc/periodic.conf and the daily status reports would include drive health information.

  • Two types of tests
  • My original abandoned attempt
  • How do you prove it works?
  • Looking at the test results
  • Failed drive to the rescue
  • smartd.conf I am using
  • supernews

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 08 Nov 2018 01:00:00 -0800
Episode 270: Ghostly Releases | BSD Now 270
OpenBSD 6.4 released, GhostBSD RC2 released, MeetBSD - the ultimate hallway track, DragonflyBSD desktop on a Thinkpad, Porting keybase to NetBSD, OpenSSH 7.9, and draft-ietf-6man-ipv6only-flag in FreeBSD.

##Headlines
OpenBSD 6.4 released

###GhostBSD 18.10 RC2 Announced

This second release candidate of GhostBSD 18.10 is the second official release of GhostBSD with TrueOS under the hood. The official desktop of GhostBSD is MATE. However, in the future, there might be an XFCE community release, but for now, there is no community release yet.

  • What has changed since RC1

  • Removed drm-stable-kmod and we will let users installed the propper drm-*-kmod

  • Douglas Joachin added libva-intel-driver libva-vdpau-driver to supports accelerated some video driver for Intel

  • Issues that got fixed

  • Bug #70 Cannot run Octopi, missing libgksu error.

  • Bug #71 LibreOffice doesn’t start because of missing libcurl.so.4

  • Bug #72 libarchive is a missing dependency

Again thanks to iXsystems, TrueOS, Joe Maloney, Kris Moore, Ken Moore, Martin Wilke, Neville Goddard, Vester “Vic” Thacker, Douglas Joachim, Alex Lyakhov, Yetkin Degirmenci and many more who helped to make the transition from FreeBSD to TrueOS smoother.

###OpenSSH 7.9 has been released and it has support for OpenSSL 1.1

Changes since OpenSSH 7.8 This is primarily a bugfix release. New Features ssh(1), sshd(8): allow most port numbers to be specified using service names from getservbyname(3) (typically /etc/services). ssh(1): allow the IdentityAgent configuration directive to accept environment variable names. This supports the use of multiple agent sockets without needing to use fixed paths. sshd(8): support signalling sessions via the SSH protocol. A limited subset of signals is supported and only for login or command sessions (i.e. not subsystems) that were not subject to a forced command via authorizedkeys or sshdconfig. bz#1424 ssh(1): support "ssh -Q sig" to list supported signature options. Also "ssh -Q help" to show the full set of supported queries. ssh(1), sshd(8): add a CASignatureAlgorithms option for the client and server configs to allow control over which signature formats are allowed for CAs to sign certificates. For example, this allows banning CAs that sign certificates using the RSA-SHA1 signature algorithm. sshd(8), ssh-keygen(1): allow key revocation lists (KRLs) to revoke keys specified by SHA256 hash. ssh-keygen(1): allow creation of key revocation lists directly from base64-encoded SHA256 fingerprints. This supports revoking keys using only the information contained in sshd(8) authentication log messages. Bugfixes ssh(1), ssh-keygen(1): avoid spurious "invalid format" errors when attempting to load PEM private keys while using an incorrect passphrase. bz#2901 sshd(8): when a channel closed message is received from a client, close the stderr file descriptor at the same time stdout is closed. This avoids stuck processes if they were waiting for stderr to close and were insensitive to stdin/out closing. bz#2863 ssh(1): allow ForwardX11Timeout=0 to disable the untrusted X11 forwarding timeout and support X11 forwarding indefinitely. Previously the behaviour of ForwardX11Timeout=0 was undefined. sshd(8): when compiled with GSSAPI support, cache supported method OIDs regardless of whether GSSAPI authentication is enabled in the main section of sshd_config. This avoids sandbox violations if GSSAPI authentication was later enabled in a Match block. bz#2107 sshd(8): do not fail closed when configured with a text key revocation list that contains a too-short key. bz#2897 ssh(1): treat connections with ProxyJump specified the same as ones with a ProxyCommand set with regards to hostname canonicalisation (i.e. don't try to canonicalise the hostname unless CanonicalizeHostname is set to 'always'). bz#2896 ssh(1): fix regression in OpenSSH 7.8 that could prevent public- key authentication using certificates hosted in a ssh-agent(1) or against sshd(8) from OpenSSH <7.8. Portability All: support building against the openssl-1.1 API (releases 1.1.0g and later). The openssl-1.0 API will remain supported at least until OpenSSL terminates security patch support for that API version. sshd(8): allow the futex(2) syscall in the Linux seccomp sandbox; apparently required by some glibc/OpenSSL combinations. sshd(8): handle getgrouplist(3) returning more than SCNGROUPSMAX groups. Some platforms consider this limit more as a guideline.

##News Roundup

###MeetBSD 2018: The Ultimate Hallway Track

Founded in Poland in 2007 and first hosted in California in 2008, MeetBSD combines formal talks with UnConference activities to provide a level of interactivity not found at any other BSD conference. The character of each MeetBSD is determined largely by its venue, ranging from Hacker Dojo in 2010 to Intel’s Santa Clara headquarters this year. The Intel SC12 building provided a beautiful auditorium and sponsors’ room, plus a cafeteria for the Friday night social event and the Saturday night FreeBSD 25th Anniversary Celebration. The formal nature of the auditorium motivated the formation of MeetBSD’s first independent Program Committee and public Call for Participation. Together these resulted in a backbone of talks presented by speakers from the USA, Canada, and Poland, combined with UnConference activities tailored to the space.

  • MeetBSD Day 0

Day Zero of MeetBSD was a FreeBSD Developer/Vendor Summit hosted in the same auditorium where the talks would take place. Like the conference itself, this event featured a mix of scheduled talks and interactive sessions. The scheduled talks were LWPMFS: LightWeight Persistent Memory Filesystem by Ravi Pokala, Evaluating GIT for FreeBSD by Ed Maste, and NUMA by Mark Johnston. Ed’s overview of the advantages and disadvantages of using Git for FreeBSD development was of the most interest to users and developers, and the discussion continued into the following two days.

  • MeetBSD Day 1

The first official day of MeetBSD 2018 was kicked off with introductions led by emcee JT Pennington and a keynote, “Using TrueOS to boot-strap your FreeBSD-based project” by Kris Moore. Kris described a new JSON-based release infrastructure that he has exercised with FreeBSD, TrueOS, and FreeNAS. Kris’ talk was followed by “Intel & FreeBSD: Better Together” by Ben Widawsky, the FreeBSD program lead at Intel, who gave an overview of Intel’s past and current efforts supporting FreeBSD. Next came lunch, followed by Kamil Rytarowski’s “Bug detecting software in the NetBSD userland: MKSANITIZER”. This was followed by 5-Minute Lightning Talks, Andrew Fengler’s “FreeBSD: What to (Not) Monitor”, and an OpenZFS Panel Discussion featuring OpenZFS experts Michael W. Lucas, Allan Jude, Alexander Motin, Pawel Dawidek, and Dan Langille. Day one concluded with a social event at the Intel cafeteria where the discussions continued into the night.

  • MeetBSD Day 2

Day Two of MeetBSD 2018 kicked off with a keynote by Michael W. Lucas entitled “Why BSD?”, where Michael detailed what makes the BSD community different and why it attracts us all. This was followed by Dr. Kirk McKusick’s “The Early Days of BSD” talk, which was followed by “DTrace/dwatch in Production” by Devin Teske. After lunch, we enjoyed “A Curmudgeon’s Language Selection Criteria: Why I Don’t Write Everything in Go, Rust, Elixir, etc” by G. Clifford Williams and, “Best practices of sandboxing applications with Capsicum” by Mariusz Zaborski. I then hosted a Virtualization Panel Discussion that featured eight developers from FreeBSD, OpenBSD, and NetBSD. We then split up for Breakout Sessions and the one on Bloomberg’s controversial article on backdoored Supermicro systems was fascinating given the experts present, all of whom were skeptical of the feasibility of the attack. The day wrapped up with a final talk, “Tales of a Daemontown Performance Peddler: Why ‘it depends’ and what you can do about it” by Nick Principe, followed by the FreeBSD 25th Anniversary Celebration.

  • Putting the “meet” in MeetBSD

I confess the other organizers and I were nervous about how well one large auditorium would suit a BSD event but the flexible personal space it gave everyone allowed for countless meetings and heated hacking that often brought about immediate results. I watched people take ideas through several iterations with the help and input of obvious and unexpected experts, all of whom were within reach. Not having to pick up and leave for a talk in another room organically resulted in essentially a series of mini hackathons that none of us anticipated but were delighted to witness, taking the “hallway track” to a whole new level. The mix of formal and UnConference activities at MeetBSD is certain to evolve. Thank you to everyone who participated with questions, Lightning Talks, and Panel participation. A huge thanks to our sponsors, including Intel for both hosting and sponsoring MeetBSD California 2018, Western Digital, Supermicro, Verisign, Jupiter Broadcasting, the FreeBSD Foundation, Bank of America Merrill Lynch, the NetBSD Foundation, and the team at iXsystems.

See you at MeetBSD 2020!

###Setup DragonflyBSD with a desktop on real hardware ThinkPad T410
+Video Demo

Linux has become too mainstream and standard BSD is a common thing now? How about DragonflyBSD which was created as a fork of FreeBSD 4.8 in conflict over system internals. This tutorial will show how to install it and set up a user-oriented desktop. It should work with DragonflyBSD, FreeBSD and probably all BSDs.
Some background: BSD was is ultimately derived from UNIX back in the days. It is not Linux even though it is similar in many ways because Linux was designed to follow UNIX principles. Seeing is believing, so check out the video of the install!
I did try two BSD distros before called GhostBSD and TrueOS and you can check out my short reviews. DragonflyBSD comes like FreeBSD bare bones and requires some work to get a desktop running.

  • Download image file and burn to USB drive or DVD

  • First installation

  • Setting up the system and installing a desktop

  • Inside the desktop

  • Install some more programs

  • How to enable sound?

  • Let’s play some free games

  • Setup WiFi

  • Power mode settings

  • More to do?

You can check out this blog post if you want a much more detailed tutorial. If you don’t mind standard BSD, get the GhostBSD distro instead which comes with a ready-made desktop xcfe or mate and many functional presets.

  • A small summary of what we got on the upside:

    • Free and open source operating system with a long history
    • Drivers worked fine including Ethernet, WiFi, video 2D & 3D, audio, etc
    • Hammer2 advanced file system
    • You are very unique if you use this OS fork
  • Some downsides:

  • Less driver and direct app support than Linux

  • Installer and desktop have some traps and quirks and require work

###Porting Keybase to NetBSD

Keybase significantly simplifies the whole keypair/PGP thing and makes what is usually a confusing, difficult experience actually rather pleasant. At its heart is an open-source command line utility that does all of the heavy cryptographic lifting. But it’s also hooked up to the network of all other Keybase users, so you don’t have to work very hard to maintain big keychains. Pretty cool!
So, this evening, I tried to get it to all work on NetBSD.
The Keybase client code base is, in my opinion, not very well architected… there exist many different Keybase clients (command line apps, desktop apps, mobile apps) and for some reason the code for all of them are seemingly in this single repository, without even using Git submodules. Not sure what that’s about.
Anyway, “go build”-ing the command line program (it’s written in Go) failed immediately because there’s some platform-specific code that just does not seem to recognize that NetBSD exists (but they do for FreeBSD and OpenBSD). Looks like the Keybase developers maintain a Golang wrapper around struct proc, which of course is different from OS to OS. So I literally just copypasted the OpenBSD wrapper, renamed it to “NetBSD”, and the build basically succeeded from there! This is of course super janky and untrustworthy, but it seems to Mostly Just Work…
I forked the GitHub repo, you can see the diff on top of keybase 2.7.3 here: bccaaf3096a
Eventually I ended up with a ~/go/bin/keybase which launches just fine. Meaning, I can main() okay. But the moment you try to do anything interesting, it looks super scary:

charlotte@sakuracity:~/go/bin ./keybase login ▶ WARNING Running in devel mode ▶ INFO Forking background server with pid=12932 ▶ ERROR unexpected error in Login: API network error: doRetry failed, attempts: 1, timeout 5s, last err: Get http://localhost:3000//api/1.0/merkle/path.json?last=3784314&loaddeleted=1&loadresetchain=1&poll=10&sighints_low=3&uid=38ae1dfa49cd6831ea2fdade5c5d0519: dial tcp [::1]:3000: connect: connection refused

There’s a few things about this error message that stuck out to me:

  • Forking a background server? What?
  • It’s trying to connect to localhost? That must be the server that doesn’t work …

Unfortunately, this nonfunctional “background server” sticks around even when a command as simple as ‘login’ command just failed:

charlotte@sakuracity:~/go/bin ps 12932 PID TTY STAT TIME COMMAND 12932 ? Ssl 0:00.21 ./keybase --debug --log-file /home/charlotte/.cache/keybase.devel/keybase.service.log service --chdir /home/charlotte/.config/keybase.devel --auto-forked

I’m not exactly sure what the intended purpose of the “background server” even is, but fortunately we can kill it and even tell the keybase command to not even spawn one:

charlotte@sakuracity:~/go/bin ./keybase help advanced | grep -- --standalone --standalone Use the client without any daemon support.

And then we can fix wanting to connect to localhost by specifying an expected Keybase API server – how about the one hosted at https://keybase.io?

charlotte@sakuracity:~/go/bin ./keybase help advanced | grep -- --server --server, -s Specify server API.

Basically, what I’m trying to say is that if you specify both of these options, the keybase command does what I expect on NetBSD:

charlotte@sakuracity:~/go/bin ./keybase --standalone -s https://keybase.io login ▶ WARNING Running in devel mode Please enter the Keybase passphrase for dressupgeekout (6+ characters): charlotte@sakuracity:~/go/bin ./keybase --standalone -s https://keybase.io id dressupgeekout ▶ WARNING Running in devel mode ▶ INFO Identifying dressupgeekout ✔ public key fingerprint: 7873 DA50 A786 9A3F 1662 3A17 20BD 8739 E82C 7F2F ✔ "dressupgeekout" on github: https://gist.github.com/0471c7918d254425835bf5e1b4bcda00 [cached 2018-10-11 20:55:21 PDT] ✔ "dressupgeekout" on reddit: https://www.reddit.com/r/KeybaseProofs/comments/9ng5qm/mykeybaseproof_redditdressupgeekout/ [cached 2018-10-11 20:55:21 PDT]

###Initial implementation of draft-ietf-6man-ipv6only-flag

This change defines the RA "6" (IPv6-Only) flag which routers may advertise, kernel logic to check if all routers on a link have the flag set and accordingly update a per-interface flag. If all routers agree that it is an IPv6-only link, etheroutputframe(), based on the interface flag, will filter out all ETHERTYPE_IP/ARP frames, drop them, and return EAFNOSUPPORT to upper layers. The change also updates ndp to show the "6" flag, ifconfig to display the IPV6_ONLY nd6 flag if set, and rtadvd to allow announcing the flag. Further changes to tcpdump (contrib code) are availble and will be upstreamed. Tested the code (slightly earlier version) with 2 FreeBSD IPv6 routers, a FreeBSD laptop on ethernet as well as wifi, and with Win10 and OSX clients (which did not fall over with the "6" flag set but not understood). We may also want to (a) implement and RX filter, and (b) over time enahnce user space to, say, stop dhclient from running when the interface flag is set. Also we might want to start IPv6 before IPv4 in the future. All the code is hidden under the EXPERIMENTAL option and not compiled by default as the draft is a work-in-progress and we cannot rely on the fact that IANA will assign the bits as requested by the draft and hence they may change. Dear 6man, you have running code. Discussed with: Bob Hinden, Brian E Carpenter

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 01 Nov 2018 04:00:00 -0700
Episode 269: Tiny Daemon Lib | BSD Now 269
FreeBSD Foundation September Update, tiny C lib for programming Unix daemons, EuroBSDcon trip reports, GhostBSD tested on real hardware, and a BSD auth module for duress.

##Headlines
FreeBSD Foundation Update, September 2018

  • MESSAGE FROM THE EXECUTIVE DIRECTOR

Dear FreeBSD Community Member, It is hard to believe that September is over. The Foundation team had a busy month promoting FreeBSD all over the globe, bug fixing in preparation for 12.0, and setting plans in motion to kick off our 4th quarter fundraising and advocacy efforts. Take a minute to see what we’ve been up to and please consider making a donation to help us continue our efforts supporting FreeBSD!

  • September 2018 Development Projects Update

In preparation for the release of FreeBSD 12.0, I have been working on investigating and fixing a backlog of kernel bug reports. Of course, this kind of work is never finished, and we will continue to make progress after the release. In the past couple of months I have fixed a combination of long-standing issues and recent regressions. Of note are a pair of UNIX domain socket bugs which had been affecting various applications for years. In particular, Chromium tabs would frequently hang unless a workaround was manually applied to the system, and the bug had started affecting recent versions of Firefox as well. Fixing these issues gave me an opportunity to revisit and extend our regression testing for UNIX sockets, which, in turn, resulted in some related bugs being identified and fixed.
Of late I have also been investigating reports of issues with ZFS, particularly, those reported on FreeBSD 11.2. A number of regressions, including a kernel memory leak and issues with ARC reclamation, have already been fixed for 12.0; investigation of other reports is ongoing. Those who closely follow FreeBSD-CURRENT know that some exciting work to improve memory usage on NUMA systems is now enabled by default. As is usually the case when new code is deployed in a diverse array of systems and workloads, a number of problems since have been identified. We are working on resolving them as soon as possible to ensure the quality of the release.
I’m passionate about maintaining FreeBSD’s stability and dependability as it continues to expand and grow new features, and I’m grateful to the FreeBSD Foundation for sponsoring this work. We depend on users to report problems to the mailing lists and via the bug tracker, so please try running the 12.0 candidate builds and help us make 12.0 a great release.

  • Fundraising Update: Supporting the Project

It’s officially Fall here at Foundation headquarters and we’re heading full-steam into our final fundraising campaign of the year. We couldn’t even have begun to reach our funding goal of $1.25 million dollars without the support from the companies who have partnered with us this year. Thank you to Verisign for becoming a Silver Partner. They now join a growing list of companies like Xiplink, NetApp, Microsoft, Tarsnap, VMware, and NeoSmart Technologies that are stepping up and showing their commitment to FreeBSD!
Funding from commercial users like these and individual users like yourself, help us continue our efforts of supporting critical areas of FreeBSD such as:

  • Operating System Improvements: Providing staff to immediately respond to urgent problems and implement new features and functionality allowing for the innovation and stability you’ve come to rely on.
  • Security: Providing engineering resources to bolster the capacity and responsiveness of the Security team providing your users with piece of mind when security issues arise.
  • Release Engineering: Continue providing a full-time release engineer, resulting in timely and reliable releases you can plan around.
  • Quality Assurance: Improving and increasing test coverage, continuous integration, and automated testing with a full-time software engineer to ensure you receive the highest quality, secure, and reliable operating system.
  • New User Experience: Improving the process and documentation for getting new people involved with FreeBSD, and supporting those people as they become integrated into the FreeBSD Community providing the resources you may need to get new folks up to speed.
  • Training: Supporting more FreeBSD training for undergraduates, graduates, and postgraduates. Growing the community means reaching people and catching their interest in systems software as early as possible and providing you with a bigger pool of candidates with the FreeBSD skills you’re looking for.
  • Face-to-Face Opportunities: Facilitating collaboration among members of the community, and building connections throughout the industry to support a healthy and growing ecosystem and make it easier for you to find resources when questions emerge .

We can continue the above work, if we meet our goal this year!
If your company uses FreeBSD, please consider joining our growing list of 2018 partners. If you haven’t made your donation yet, please consider donating today. We are indebted to the individual donors, and companies listed above who have already shown their commitment to open source.
Thank you for supporting FreeBSD and the Foundation!

  • September 2018 Release Engineering Update

The FreeBSD Release Engineering team continued working on the upcoming 12.0 RELEASE. At present, the 12.0 schedule had been adjusted by one week to allow for necessary works-in-progress to be completed.
Of note, one of the works-in-progress includes updating OpenSSL from 1.0.2 to 1.1.1, in order to avoid breaking the application binary interface (ABI) on an established stable branch.
Due to the level of non-trivial intrusiveness that had already been discovered and addressed in a project branch of the repository, it is possible (but not yet definite) that the schedule will need to be adjusted by another week to allow more time for larger and related updates for this particular update.
Should the 12.0-RELEASE schedule need to be adjusted at any time during the release cycle, the schedule on the FreeBSD project website will be updated accordingly. The current schedule is available at:
https://www.freebsd.org/releases/12.0R/schedule.html

  • BSDCam 2018 Trip Report: Marie Helene Kvello-Aune

I’d like to start by thanking the FreeBSD Foundation for sponsoring my trip to BSDCam(bridge) 2018. I wouldn’t have managed to attend otherwise. I’ve used FreeBSD in both personal and professional deployments since the year 2000, and over the last few years I have become more involved with development and documentation.
I arrived in Gatwick, London at midnight. On Monday, August 13, I took the train to Cambridge, and decided to do some touristy activities as I walked from the train station to Churchill College. I ran into Allan outside the hotel right before the sky decided it was time for a heavy rainfall. Monday was mostly spent settling in, recouping after travel, and hanging out with Allan, Brad, Will and Andy later in the afternoon/evening. Read more…

  • Continuous Integration Update

The FreeBSD Foundation has sponsored the development of the Project’s continuous integration system, available at https://ci.FreeBSD.org, since June. Over the summer, we improved both the software and hardware infrastructure, and also added some new jobs for extending test coverage of the -CURRENT and -STABLE branches. Following are some highlights.

  • New Hardware

The Foundation purchased 4 new build machines for scaling up the computation power for the various test jobs. These newer, faster machines substantially speed up the time it takes to test amd64 builds, so that failing changes can be identified more quickly. Also, in August, we received a donation of 2 PINE A64-LTS boards from PINE64.org, which will be put in the hardware test lab as one part of the continuous tests.

  • CI Staging Environment

We used hardware from a previous generation CI system to build a staging environment for the CI infrastructure, which is available at
https://ci-dev.freebsd.org. It executes the configurations and scripts from the “staging” branch of the FreeBSD-CI repository, and the development feature branches. We also use it to experiment with the new version of the jenkins server and plugins. Having a staging environment avoids affecting the production CI environment, reducing downtime.

  • Mail Notification

In July, we turned on failure notification for all the kernel and world build jobs. Committers will receive email containing the build information and failure log to inform them of possible problems with their modification on certain architectures. For amd64 of the -CURRENT branch, we also enabled the notification on failing regression test cases. Currently mail is sent only to the individual committers, but with help from postmaster team, we have created a dev-ci mailing list and will soon be also sending notifications there.

  • New Test Job

In August, we updated the embedded script of the virtual machine image. Originally it only executed pre-defined tests, but now this behavior can be modified by the data on the attached disk. This mechanism is used for adding new ZFS tests jobs. We are also working on analyzing and fixing the failing and skipped test cases.

  • Work in Progress

In August and September, we had two developer summits, one in Cambridge, UK and one in Bucharest, Romania. In these meetings, we discussed running special tests, such as ztest, which need a longer run time. We also planned the network testing for TCP/IP stack

###Daemonize - a Tiny C Library for Programming the UNIX Daemons

Whatever they say, writing System-V style UNIX daemons is hard. One has to follow many rules to make a daemon process behave correctly on diverse UNIX flavours. Moreover, debugging such a code might be somewhat tricky. On the other hand, the process of daemon initialisation is rigid and well defined so the corresponding code has to be written and debugged once and later can be reused countless number of times.
Developers of BSD UNIX were very aware of this, as there a C library function daemon() was available starting from version 4.4. The function, although non-standard, is present on many UNIXes. Unfortunately, it does not follow all the required steps to reliably run a process in the background on systems which follow System-V semantics (e.g. Linux). The details are available at the corresponding Linux man page. The main problem here, as I understand it, is that daemon() does not use the double-forking technique to avoid the situation when zombie processes appear.
Whenever I encounter a problem like this one, I know it is time to write a tiny C library which solves it. This is exactly how ‘daemonize’ was born (GitHub mirror). The library consists of only two files which are meant to be integrated into the source tree of your project. Recently I have updated the library and realised that it would be good to describe how to use it on this site.
If for some reason you want to make a Windows service, I have a battle tested template code for you as well.

  • System-V Daemon Initialisation Procedure

To make discussion clear we shall quote the steps which have to be performed during a daemon initialisation (according to daemon(7) manual page on Linux). I do it to demonstrate that this task is more tricky than one might expect.

  • So, here we go:

  • Close all open file descriptors except standard input, output, and error (i.e. the first three file descriptors 0, 1, 2). This ensures that no accidentally passed file descriptor stays around in the daemon process. On Linux, this is best implemented by iterating through /proc/self/fd, with a fallback of iterating from file descriptor 3 to the value returned by getrlimit() for RLIMITNOFILE.

  • Reset all signal handlers to their default. This is best done by iterating through the available signals up to the limit of _NSIG and resetting them to SIGDFL.

  • Reset the signal mask using sigprocmask().

  • Sanitize the environment block, removing or resetting environment variables that might negatively impact daemon runtime.

  • Call fork(), to create a background process.

  • In the child, call setsid() to detach from any terminal and create an independent session.

  • In the child, call fork() again, to ensure that the daemon can never re-acquire a terminal again.

  • Call exit() in the first child, so that only the second child (the actual daemon process) stays around. This ensures that the daemon process is re-parented to init/PID 1, as all daemons should be.

  • In the daemon process, connect /dev/null to standard input, output, and error.

  • In the daemon process, reset the umask to 0, so that the file modes passed to open(), mkdir() and suchlike directly control the access mode of the created files and directories.

  • In the daemon process, change the current directory to the root directory (/), in order to avoid that the daemon involuntarily blocks mount points from being unmounted.

  • In the daemon process, write the daemon PID (as returned by getpid()) to a PID file, for example /run/foobar.pid (for a hypothetical daemon “foobar”) to ensure that the daemon cannot be started more than once. This must be implemented in race-free fashion so that the PID file is only updated when it is verified at the same time that the PID previously stored in the PID file no longer exists or belongs to a foreign process.

  • In the daemon process, drop privileges, if possible and applicable.

  • From the daemon process, notify the original process started that initialization is complete. This can be implemented via an unnamed pipe or similar communication channel that is created before the first fork() and hence available in both the original and the daemon process.

  • Call exit() in the original process. The process that invoked the daemon must be able to rely on that this exit() happens after initialization is complete and all external communication channels are established and accessible.

The discussed library does most of the above-mentioned initialisation steps as it becomes immediately evident that implementation details for some of them heavily dependent on the internal logic of an application itself, so it is not possible to implement them in a universal library. I believe it is not a flaw, though, as the missed parts are safe to implement in an application code.

  • The Library’s Application Programming Interface

The generic programming interface was loosely modelled after above-mentioned BSD’s daemon() function. The library provides two user available functions (one is, in fact, implemented on top of the other) as well as a set of flags to control a daemon creation behaviour.

  • Conclusion

The objective of the library is to hide all the trickery of programming a daemon so you could concentrate on the more creative parts of your application. I hope it does this well.
If you are not only interested in writing a daemon, but also want to make yourself familiar with the techniques which are used to accomplish that, the source code is available. Moreover, I would advise anyone, who starts developing for a UNIX environment to do that, as it shows many intricacies of programming for these platforms.

##News Roundup
EuroBSDCon 2018 travel report and obligatory pics

This was my first big BSD conference. We also planned - planned might be a big word - thought about doing a devsummit on Friday. Since the people who were in charge of that had a change of plans, I was sure it’d go horribly wrong.
The day before the devsummit and still in the wrong country, I mentioned the hours and venue on the wiki, and booked a reservation for a restaurant.
It turns out that everything was totally fine, and since the devsummit was at the conference venue (that was having tutorials that day), they even had signs pointing at the room we were given. Thanks EuroBSDCon conference organizers!
At the devsummit, we spent some time hacking. A few people came with “travel laptops” without access to anything, like Riastradh, so I gave him access to my own laptop. This didn’t hold very long and I kinda forgot about it, but for a few moments he had access to a NetBSD source tree and an 8 thread, 16GB RAM machine with which to build things.
We had a short introduction and I suggested we take some pictures, so here’s the ones we got. A few people were concerned about privacy, so they’re not pictured. We had small team to hold the camera :-)
At the actual conference days, I stayed at the speaker hotel with the other speakers. I’ve attempted to make conversation with some visibly FreeBSD/OpenBSD people, but didn’t have plans to talk about anything, so there was a lot of just following people silently.
Perhaps for the next conference I’ll prepare a list of questions to random BSD people and then very obviously grab a piece of paper and ask, “what was…”, read a bit from it, and say, “your latest kernel panic?”, I’m sure it’ll be a great conversation starter.
At the conference itself, was pretty cool to have folks like Kirk McKusick give first person accounts of some past events (Kirk gave a talk about governance at FreeBSD), or the second keynote by Ron Broersma.
My own talk was hastily prepared, it was difficult to bring the topic together into a coherent talk. Nevertheless, I managed to talk about stuff for a while 40 minutes, though usually I skip over so many details that I have trouble putting together a sufficiently long talk.
I mentioned some of my coolest bugs to solve (I should probably make a separate article about some!). A few people asked for the slides after the talk, so I guess it wasn’t totally incoherent.
It was really fun to meet some of my favourite NetBSD people. I got to show off my now fairly well working laptop (it took a lot of work by all of us!).
After the conference I came back with a conference cold, and it took a few days to recover from it. Hopefully I didn’t infect too many people on the way back.

###GhostBSD tested on real hardware T410 – better than TrueOS?

You might have heard about FreeBSD which is ultimately derived from UNIX back in the days. It is not Linux even though it is similar in many ways because Linux was designed to follow UNIX principles. Seeing is believing, so check out the video of the install and some apps as well!

Nowadays if you want some of that BSD on your personal desktop how to go about? Well there is a full package or distro called GhostBSD which is based on FreeBSD current with a Mate or XFCE desktop preconfigured. I did try another package called TrueOS before and you can check out my blog post as well.

Let’s give it a try on my Lenovo ThinkPad T410. You can download the latest version from ghostbsd.org. Creating a bootable USB drive was surprisingly difficult as rufus did not work and created a corrupted drive. You have to follow this procedure under Windows: download the 2.5GB .iso file and rename the extension to .img. Download Win32 Disk imager and burn the img file to an USB drive and boot from it. You will be able to start a live session and use the onboard setup to install GhostBSD unto a disk.

I did encounter some bugs or quirks along the way. The installer failed the first time for some unknown reason but worked on the second attempt. The first boot stopped upon initialization of the USB3 ports (the T410 does not have USB3) but I could use some ‘exit’ command line magic to continue. The second boot worked fine. Audio was only available through headphones, not speakers but that could partially be fixed using the command line again. Lot’s of installed apps did not show up in the start menu and on goes the quirks list.

Overall it is still better than TrueOS for me because drivers did work very well and I could address most of the existing bugs.

  • On the upside:

  • Free and open source FreeBSD package ready to go

  • Mate or XFCE desktop (Mate is the only option for daily builds)

  • Drivers work fine including LAN, WiFi, video 2D & 3D, audio, etc

  • UFS or ZFS advanced file systems available

  • Some downsides:

  • Less driver and direct app support than Linux

  • Installer and desktop have some quirks and bugs

  • App-store is cumbersome, inferior to TrueOS

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Wed, 24 Oct 2018 02:00:00 -0700
Episode 268: Netcat Demystified | BSD Now 268
6 metrics for zpool performance, 2FA with ssh on OpenBSD, ZFS maintaining file type information in dirs, everything old is new again, netcat demystified, and more.

##Headlines
Six Metrics for Measuring ZFS Pool Performance Part 1

The layout of a ZFS storage pool has a significant impact on system performance under various workloads. Given the importance of picking the right configuration for your workload and the fact that making changes to an in-use ZFS pool is far from trivial, it is important for an administrator to understand the mechanics of pool performance when designing a storage system.

  • To quantify pool performance, we will consider six primary metrics:
  • Read I/O operations per second (IOPS)
  • Write IOPS
  • Streaming read speed
  • Streaming write speed
  • Storage space efficiency (usable capacity after parity versus total raw capacity)
  • Fault tolerance (maximum number of drives that can fail before data loss)
  • For the sake of comparison, we’ll use an example system with 12 drives, each one sized at 6TB, and say that each drive does 100MB/s streaming reads and writes and can do 250 read and write IOPS. We will visualize how the data is spread across the drives by writing 12 multi-colored blocks, shown below. The blocks are written to the pool starting with the brown block on the left (number one), and working our way to the pink block on the right (number 12).

Note that when we calculate data rates and IOPS values for the example system, they are only approximations. Many other factors can impact pool access speeds for better (compression, caching) or worse (poor CPU performance, not enough memory).
There is no single configuration that maximizes all six metrics. Like so many things in life, our objective is to find an appropriate balance of the metrics to match a target workload. For example, a cold-storage backup system will likely want a pool configuration that emphasizes usable storage space and fault tolerance over the other data-rate focused metrics.
Let’s start with a quick review of ZFS storage pools before diving into specific configuration options. ZFS storage pools are comprised of one or more virtual devices, or vdevs. Each vdev is comprised of one or more storage providers, typically physical hard disks. All disk-level redundancy is configured at the vdev level. That is, the RAID layout is set on each vdev as opposed to on the storage pool. Data written to the storage pool is then striped across all the vdevs. Because pool data is striped across the vdevs, the loss of any one vdev means total pool failure. This is perhaps the single most important fact to keep in mind when designing a ZFS storage system. We will circle back to this point in the next post, but keep it in mind as we go through the vdev configuration options.
Because storage pools are made up of one or more vdevs with the pool data striped over the top, we’ll take a look at pool configuration in terms of various vdev configurations. There are three basic vdev configurations: striping, mirroring, and RAIDZ (which itself has three different varieties). The first section will cover striped and mirrored vdevs in this post; the second post will cover RAIDZ and some example scenarios.
A striped vdev is the simplest configuration. Each vdev consists of a single disk with no redundancy. When several of these single-disk, striped vdevs are combined into a single storage pool, the total usable storage space would be the sum of all the drives. When you write data to a pool made of striped vdevs, the data is broken into small chunks called “blocks” and distributed across all the disks in the pool. The blocks are written in “round-robin” sequence, meaning after all the disks receive one row of blocks, called a stripe, it loops back around and writes another stripe under the first. A striped pool has excellent performance and storage space efficiency, but absolutely zero fault tolerance. If even a single drive in the pool fails, the entire pool will fail and all data stored on that pool will be lost.
The excellent performance of a striped pool comes from the fact that all of the disks can work independently for all read and write operations. If you have a bunch of small read or write operations (IOPS), each disk can work independently to fetch the next block. For streaming reads and writes, each disk can fetch the next block in line synchronized with its neighbors. For example, if a given disk is fetching block n, its neighbor to the left can be fetching block n-1, and its neighbor to the right can be fetching block n+1. Therefore, the speed of all read and write operations as well as the quantity of read and write operations (IOPS) on a striped pool will scale with the number of vdevs. Note here that I said the speeds and IOPS scale with the number of vdevs rather than the number of drives; there’s a reason for this and we’ll cover it in the next post when we discuss RAID-Z.
Here’s a summary of the total pool performance (where N is the number of disks in the pool):

  • N-wide striped:
  • Read IOPS: N * Read IOPS of a single drive
  • Write IOPS: N * Write IOPS of a single drive
  • Streaming read speed: N * Streaming read speed of a single drive
  • Streaming write speed: N * Streaming write speed of a single drive
  • Storage space efficiency: 100%
  • Fault tolerance: None!

Let’s apply this to our example system, configured with a 12-wide striped pool:

  • 12-wide striped:
  • Read IOPS: 3000
  • Write IOPS: 3000
  • Streaming read speed: 1200 MB/s
  • Streaming write speed: 1200 MB/s
  • Storage space efficiency: 72 TB
  • Fault tolerance: None!
  • Below is a visual depiction of our 12 rainbow blocks written to this pool configuration:

The blocks are simply striped across the 12 disks in the pool. The LBA column on the left stands for “Logical Block Address”. If we treat each disk as a column in an array, each LBA would be a row. It’s also easy to see that if any single disk fails, we would be missing a color in the rainbow and our data would be incomplete. While this configuration has fantastic read and write speeds and can handle a ton of IOPS, the data stored on the pool is very vulnerable. This configuration is not recommended unless you’re comfortable losing all of your pool’s data whenever any single drive fails.
A mirrored vdev consists of two or more disks. A mirrored vdev stores an exact copy of all the data written to it on each one of its drives. Traditional RAID-1 mirrors usually only support two drive mirrors, but ZFS allows for more drives per mirror to increase redundancy and fault tolerance. All disks in a mirrored vdev have to fail for the vdev, and thus the whole pool, to fail. Total storage space will be equal to the size of a single drive in the vdev. If you’re using mismatched drive sizes in your mirrors, the total size will be that of the smallest drive in the mirror.
Streaming read speeds and read IOPS on a mirrored vdev will be faster than write speeds and IOPS. When reading from a mirrored vdev, the drives can “divide and conquer” the operations, similar to what we saw above in the striped pool. This is because each drive in the mirror has an identical copy of the data. For write operations, all of the drives need to write a copy of the data, so the mirrored vdev will be limited to the streaming write speed and IOPS of a single disk.

Here’s a summary:

  • N-way mirror:

  • Read IOPS: N * Read IOPS of a single drive

  • Write IOPS: Write IOPS of a single drive

  • Streaming read speed: N * Streaming read speed of a single drive

  • Streaming write speed: Streaming write speed of a single drive

  • Storage space efficiency: 50% for 2-way, 33% for 3-way, 25% for 4-way, etc. [(N-1)/N]

  • Fault tolerance: 1 disk per vdev for 2-way, 2 for 3-way, 3 for 4-way, etc. [N-1]

  • For our first example configuration, let’s do something ridiculous and create a 12-way mirror. ZFS supports this kind of thing, but your management probably will not.

  • 1x 12-way mirror:

  • Read IOPS: 3000

  • Write IOPS: 250

  • Streaming read speed: 1200 MB/s

  • Streaming write speed: 100 MB/s

  • Storage space efficiency: 8.3% (6 TB)

  • Fault tolerance: 11

As we can clearly see from the diagram, every single disk in the vdev gets a full copy of our rainbow data. The chainlink icons between the disk labels in the column headers indicate the disks are part of a single vdev. We can lose up to 11 disks in this vdev and still have a complete rainbow. Of course, the data takes up far too much room on the pool, occupying a full 12 LBAs in the data array.

Obviously, this is far from the best use of 12 drives. Let’s do something a little more practical and configure the pool with the ZFS equivalent of RAID-10. We’ll configure six 2-way mirror vdevs. ZFS will stripe the data across all 6 of the vdevs. We can use the work we did in the striped vdev section to determine how the pool as a whole will behave. Let’s first calculate the performance per vdev, then we can work on the full pool:

  • 1x 2-way mirror:

  • Read IOPS: 500

  • Write IOPS: 250

  • Streaming read speed: 200 MB/s

  • Streaming write speed: 100 MB/s

  • Storage space efficiency: 50% (6 TB)

  • Fault tolerance: 1

  • Now we can pretend we have 6 drives with the performance statistics listed above and run them through our striped vdev performance calculator to get the total pool’s performance:

  • 6x 2-way mirror:

  • Read IOPS: 3000

  • Write IOPS: 1500

  • Streaming read speed: 3000 MB/s

  • Streaming write speed: 1500 MB/s

  • Storage space efficiency: 50% (36 TB)

  • Fault tolerance: 1 per vdev, 6 total

  • Again, we will examine the configuration from a visual perspective:

Each vdev gets a block of data and ZFS writes that data to all of (or in this case, both of) the disks in the mirror. As long as we have at least one functional disk in each vdev, we can retrieve our rainbow. As before, the chain link icons denote the disks are part of a single vdev. This configuration emphasizes performance over raw capacity but doesn’t totally disregard fault tolerance as our striped pool did. It’s a very popular configuration for systems that need a lot of fast I/O. Let’s look at one more example configuration using four 3-way mirrors. We’ll skip the individual vdev performance calculation and go straight to the full pool:

  • 4x 3-way mirror:
  • Read IOPS: 3000
  • Write IOPS: 1000
  • Streaming read speed: 3000 MB/s
  • Streaming write speed: 400 MB/s
  • Storage space efficiency: 33% (24 TB)
  • Fault tolerance: 2 per vdev, 8 total

While we have sacrificed some write performance and capacity, the pool is now extremely fault tolerant. This configuration is probably not practical for most applications and it would make more sense to use lower fault tolerance and set up an offsite backup system.
Striped and mirrored vdevs are fantastic for access speed performance, but they either leave you with no redundancy whatsoever or impose at least a 50% penalty on the total usable space of your pool. In the next post, we will cover RAIDZ, which lets you keep data redundancy without sacrificing as much storage space efficiency. We’ll also look at some example workload scenarios and decide which layout would be the best fit for each.

###2FA with ssh on OpenBSD

Five years ago I wrote about using a yubikey on OpenBSD. The only problem with doing this is that there’s no validation server available on OpenBSD, so you need to use a different OTP slot for each machine. (You don’t want to risk a replay attack if someone succeeds in capturing an OTP on one machine, right?) Yubikey has two OTP slots per device, so you would need a yubikey for every two machines with which you’d like to use it. You could use a bastion—and use only one yubikey—but I don’t like the SPOF aspect of a bastion. YMMV.
After I played with TOTP, I wanted to use them as a 2FA for ssh. At the time of writing, we can’t do that using only the tools in base. This article focuses on OpenBSD; if you use another operating system, here are two handy links.

  • SEED CONFIGURATION

The first thing we need to do is to install the software which will be used to verify the OTPs we submit.

# pkgadd loginoath

We need to create a secret - aka, the seed - that will be used to calculate the Time-based One-Time Passwords. We should make sure no one can read or change it.

$ openssl rand -hex 20 > ~/.totp-key
$ chmod 400 ~/.totp-key

Now we have a hexadecimal key, but apps usually want a base32 secret. I initially wrote a small script to do the conversion.
While writing this article, I took the opportunity to improve it. When I initially wrote this utility for my use, python-qrcode hadn’t yet been imported to the OpenBSD ports/packages system. It’s easy to install now, so let’s use it.
Here’s the improved version. It will ask for the hex key and output the secret as a base32-encoded string, both with and without spacing so you can copy-paste it into your password manager or easily retype it. It will then ask for the information needed to generate a QR code. Adding our new OTP secret to any mobile app using the QR code will be super easy!

  • SYSTEM CONFIGURATION

We can now move to the configuration of the system to put our new TOTP to use. As you might guess, it’s going to be quite close to what we did with the yubikey.
We need to tweak login.conf. Be careful and keep a root shell open at all times. The few times I broke my OpenBSD were because I messed with login.conf without showing enough care.

  • SSHD CONFIGURATION

Again, keeping a root shell around decreases the risk of losing access to the system and being locked outside.
A good standard is to use PasswordAuthentication no and to use public key only. Except… have a guess what the P stands for in TOTP. Yes, congrats, you guessed it!
We need to switch to PasswordAuthentication yes. However, if we made this change alone, sshd would then accept a public key OR a password (which are TOTP because of our login.conf). 2FA uses both at the same time.
To inform sshd we intend to use both, we need to set AuthenticationMethods publickey,password. This way, the user trying to login will first need to perform the traditional publickey authentication. Once that’s done, ssh will prompt for a password and the user will need to submit a valid TOTP for the system.
We could do this the other way around, but I think bots could try passwords, wasting resources. Evaluated in this order, failing to provide a public key leads to sshd immediately declining your attempt.

  • IMPROVING SECURITY WITHOUT IMPACTING UX

My phone has a long enough password that most of the time, I fail to type it correctly on the first try. Of course, if I had to unlock my phone, launch my TOTP app and use my keyboard to enter what I see on my phone’s screen, I would quickly disable 2FA.
To find a balance, I have whitelisted certain IP addresses and users. If I connect from a particular IP address or as a specific user, I don’t want to go through 2FA. For some users, I might not even enable 2FA.
To sum up, we covered how to create a seed, how to perform a hexadecimal to base32 conversion and how to create a QR code for mobile applications. We configured the login system with login.conf so that ssh authentication uses the TOTP login system, and we told sshd to ask for both the public key and the Time-based One-Time Password. Now you should be all set to use two-factor ssh authentication on OpenBSD!

##News Roundup
How ZFS maintains file type information in directories

As an aside in yesterday’s history of file type information being available in Unix directories, I mentioned that it was possible for a filesystem to support this even though its Unix didn’t. By supporting it, I mean that the filesystem maintains this information in its on disk format for directories, even though the rest of the kernel will never ask for it. This is what ZFS does.
The easiest way to see that ZFS does this is to use zdb to dump a directory. I’m going to do this on an OmniOS machine, to make it more convincing, and it turns out that this has some interesting results. Since this is OmniOS, we don’t have the convenience of just naming a directory in zdb, so let’s find the root directory of a filesystem, starting from dnode 1 (as seen before).

# zdb -dddd fs3-corestaff-01/h/281 1
Dataset [....]
[...]
microzap: 512 bytes, 4 entries
[...]
ROOT = 3

# zdb -dddd fs3-corestaff-01/h/281 3
Object lvl iblk dblk dsize lsize %full type
3 1 16K 1K 8K 1K 100.00 ZFS directory
[...]
microzap: 1024 bytes, 8 entries

RESTORED = 4396504 (type: Directory)
ckstst = 12017 (type: not specified)
ckstst3 = 25069 (type: Directory)
.demo-file = 5832188 (type: Regular File)
.peergroup = 12590 (type: not specified)
cks = 5 (type: not specified)
cksimap1 = 5247832 (type: Directory)
.diskuse = 12016 (type: not specified)
ckstst2 = 12535 (type: not specified)

This is actually an old filesystem (it dates from Solaris 10 and has been transferred around with ‘zfs send | zfs recv’ since then), but various home directories for real and test users have been created in it over time (you can probably guess which one is the oldest one). Sufficiently old directories and files have no file type information, but more recent ones have this information, including .demo-file, which I made just now so this would have an entry that was a regular file with type information.
Once I dug into it, this turned out to be a change introduced (or activated) in ZFS filesystem version 2, which is described in ‘zfs upgrade -v’ as ‘enhanced directory entries’. As an actual change in (Open)Solaris, it dates from mid 2007, although I’m not sure what Solaris release it made it into. The upshot is that if you made your ZFS filesystem any time in the last decade, you’ll have this file type information in your directories.
How ZFS stores this file type information is interesting and clever, especially when it comes to backwards compatibility. I’ll start by quoting the comment from zfs_znode.h:

/*
* The directory entry has the type (currently unused on
* Solaris) in the top 4 bits, and the object number in
* the low 48 bits. The "middle" 12 bits are unused.
*/

In yesterday’s entry I said that Unix directory entries need to store at least the filename and the inode number of the file. What ZFS is doing here is reusing the 64 bit field used for the ‘inode’ (the ZFS dnode number) to also store the file type, because it knows that object numbers have only a limited range. This also makes old directory entries compatible, by making type 0 (all 4 bits 0) mean ‘not specified’. Since old directory entries only stored the object number and the object number is 48 bits or less, the higher bits are guaranteed to be all zero.
The reason this needed a new ZFS filesystem version is now clear. If you tried to read directory entries with file type information on a version of ZFS that didn’t know about them, the old version would likely see crazy (and non-existent) object numbers and nothing would work. In order to even read a ‘file type in directory entries’ filesystem, you need to know to only look at the low 48 bits of the object number field in directory entries.

###Everything old is new again

Just because KDE4-era software has been deprecated by the KDE-FreeBSD team in the official ports-repository, doesn’t mean we don’t care for it while we still need to. KDE4 was released on January 11th, 2008 — I still have the T-shirt — which was a very different C++ world than what we now live in. Much of the code pre-dates the availability of C11 — certainly the availability of compilers with C11 support. The language has changed a great deal in those ten years since the original release.
The platforms we run KDE code on have, too — FreeBSD 12 is a long way from the FreeBSD 6 or 7 that were current at release (although at the time, I was more into OpenSolaris). In particular, since then the FreeBSD world has switched over to Clang, and FreeBSD current is experimenting with Clang 7. So we’re seeing KDE4-era code being built, and running, on FreeBSD 12 with Clang 7. That’s a platform with a very different idea of what constitutes correct code, than what the code was originally written for. (Not quite as big a difference as Helio’s KDE1 efforts, though)
So, while we’re counting down to removing KDE4 from the FreeBSD ports tree, we’re also going through and fixing it to work with Clang 7, which defaults to a newer C++ standard and which is quite picky about some things. Some time in the distant past, when pointers were integers and NULL was zero, there was some confusion about booleans. So there’s lots of code that does list.contains(element) > 0 … this must have been a trick before booleans were a supported type in all our compilers. In any case it breaks with Clang 7, since contains() returns a QBool which converts to a nullptr (when false) which isn’t comparable to the integer 0. Suffice to say I’ve spent more time reading KDE4-era code this month, than in the past two years.
However, work is proceeding apace, so if you really really want to, you can still get your old-school kicks on a new platform. Because we care about packaging things right, even when we want to get rid of it.

###OpenBSD netcat demystified

Owing to its versatile functionalities, netcat earns the reputation as “TCP/IP Swiss army knife”. For example, you can create a simple chat app using netcat:

  • (1) Open a terminal and input following command:

# nc -l 3003

This means a netcat process will listen on 3003 port in this machine (the IP address of current machine is 192.168.35.176).

  • (2) Connect aforemontioned netcat process in another machine, and send a greeting:

# nc 192.168.35.176 3003
hello

Then in the first machine’s terminal, you will see the “hello” text:

# nc -l 3003
hello

A primitive chatroom is built successfully. Very cool! Isn’t it? I think many people can’t wait to explore more features of netcatnow. If you are among them, congratulations! This tutorial may be the correct place for you.
In the following parts, I will delve into OpenBSD’s netcatcode to give a detailed anatomy of it. The reason of picking OpenBSD’s netcat rather than others’ is because its code repository is small (~2000 lines of code) and neat. Furthermore, I also hope this little book can assist you learn more socket programming knowledge not just grasping usage of netcat.
We’re all set. Let’s go!

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Tue, 16 Oct 2018 22:00:00 -0700
Episode 267: Absolute FreeBSD | BSD Now 267
We have a long interview with fiction and non-fiction author Michael W. Lucas for you this week as well as questions from the audience.

##Headlines
Interview - Michael W. Lucas - mwlucas@michaelwlucas.com / @mwlauthor

  • BR: [Welcome Back]
  • AJ: What have you been doing since last we talked to you [ed, ssh, and af3e]
  • BR: Tell us more about AF3e
  • AJ: How did the first Absolute FreeBSD come about?
  • BR: Do you have anything special planned for MeetBSD?
  • AJ: What are you working on now? [FM:Jails, Git sync Murder]
  • BR: What are your plans for next year?
  • AJ: How has SEMIBug been going?

Auction at https://mwl.io
Patreon Link:

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Wed, 10 Oct 2018 03:00:00 -0700
Episode 266: File Type History | BSD Now 266
Running OpenBSD/NetBSD on FreeBSD using grub2-bhyve, vermaden’s FreeBSD story, thoughts on OpenBSD on the desktop, history of file type info in Unix dirs, Multiboot a Pinebook KDE neon image, and more.

##Headlines
OpenBSD/NetBSD on FreeBSD using grub2-bhyve

When I was writing a blog post about the process title, I needed a couple of virtual machines with OpenBSD, NetBSD, and Ubuntu. Before that day I mainly used FreeBSD and Windows with bhyve. I spent some time trying to set up an OpenBSD using bhyve and UEFI as described here. I had numerous problems trying to use it, and this was the day I discovered the grub2-bhyve tool, and I love it!
The grub2-bhyve allows you to load a kernel using GRUB bootloader. GRUB supports most of the operating systems with a standard configuration, so exactly the same method can be used to install NetBSD or Ubuntu. First, let’s install grub2-bhyve on our FreeBSD box:

# pkg install grub2-bhyve

To run grub2-bhyve we need to provide at least the name of the VM. In bhyve, if the memsize is not specified the default VM is created with 256MB of the memory.

# grub-bhyve test
GNU GRUB version 2.00
Minimal BASH-like line editing is supported. For the first word, TAB lists possible command
completions. Anywhere else TAB lists possible device or file completions.


grub>

After running grub-bhyve command we will enter the GRUB loader. If we type the ls command, we will see all the available devices. In the case of the grub2-bhyve there is one additional device called “(host)” that is always available and allows the host filesystem to be accessed. We can list files under that device.

grub> ls
(host)
grub> ls (host)/
libexec/ bin/ usr/ bhyve/ compat/ tank/ etc/ boot/ net/ entropy proc/ lib/ root/ sys/ mnt/ rescue/ tmp/ home/ sbin/ media/ jail/ COPYRIGHT var/ dev/
grub>

To exit console simply type ‘reboot’. I would like to install my new operating system under a ZVOL ztank/bhyve/post. On another terminal, we create:

# zfs create -V 10G ztank/bhyve/post

If you don’t use ZFS for some crazy reason you can also create a raw blob using the truncate(1) command.

# truncate -s 10G post.img

I recommend installing an operating system from the disk image (installXX.fs for OpenBSD and NetBSD-X.X-amd64-install.img for NetBSD). Now we need to create a device map for a GRUB.

cat > /tmp/post.map << EOF
(hd0) /directory/to/disk/image
(hd1) /dev/zvol/ztank/bhyve/post
EOF

The mapping files describe the names for files in the GRUB. In our case under hd0 we will have an installation image and in hd1 we will have our ZVOL/blob. You can also try to use an ISO image then instead of using hd0 device name use a cd0. When we will run the grub-bhyve command we will see two additional devices.

# grub-bhyve -m /tmp/post.map post
grub> ls
(hd0) (hd0,msdos4) (hd0,msdos1) (hd0,openbsd9) (hd0,openbsd1) (hd1) (host)

The hd0 (in this example OpenBSD image) contains multiple partitions. We can check what is on it.

grub> ls (hd0,msdos4)/
boot bsd 6.4/ etc/

And this is the partition that contains a kernel. Now we can set a root device, load an OpenBSD kernel and boot:

grub> set root=(hd0,msdos4)
grub> kopenbsd -h com0 -r sd0a /bsd
grub> boot

After that, we can run bhyve virtual machine. In my case it is:

# bhyve -c 1 -w -u -H \
-s 0,amd_hostbridge \
-s 3,ahci-hd,/directory/to/disk/image \
-s 4,ahci-hd,/dev/zvol/ztank/bhyve/post \
-s 31,lpc -l com1,stdio \
post

Unfortunately explaining the whole bhyve(8) command line is beyond this article. After installing the operating system remove hd0 from the mapping file and the image from the bhyve(8) command. If you don’t want to type all those GRUB commands, you can simply redirect them to the standard input.

cat << EOF | grub-bhyve -m /tmp/post.map -M 512 post
set root=(hd0,4)
kopenbsd -h com0 -r sd0a /bsd
boot
EOF

###My FreeBSD Story

My first devices/computers/consoles (not at the same time) that I remember were Atari 2600 and Pegasus console which was hardware clone of the Nintendo NES.
Back then I did not even knew that it was Atari 2600 as I referred to it as Video Computer System … and I did not even knew any english by then. It took me about two decades to get to know (by accident) that this Video Computer System was Atari 2600
Then I got AMIGA 600 computer (or should I say my parents bought it for me) which served both for playing computer games and also other activities for the first time. AMIGA is the computer that had the greatest influence on me, as it was the first time I studied the books about Amiga Workbench operating system and learned commands from Amiga Shell terminal. I loved the idea of Ram Disk icon/directory on the desktop that allowed me to transparently put any things in system memory. I still miss that concept on today’s desktop systems … and I still remember how dismal I was when I watched Amiga Deathbed Vigil movie.
At the end of 1998 I got my first PC that of course came with Windows and that computer served both as gaming machine and as well as typical tool. One time I dig into the internals with Windows Registry (which left me disgusted by its concepts and implementation) and its limited command line interface provided by CMD.EXE executable. I remember that the heart of this box was not the CPU or the motherboard but the graphics accelerator – the legendary 3Dfx Voodoo card. This company (3Dfx) – their attitude and philosophy – also left solid fingerprint on my way. Like AMIGA did.
After ‘migration’ from AMIGA to PC it never again ‘felt right’. The games were cool but the Windows system was horrible. Time has passed and different Windows versions and hardware modifications took place. Windows XP felt really heavy at that time, not to mention Windows 2000 for example with even bigger hardware requirements. I also do not understand all the hate about Windows ME. It crashed with the same frequency as Windows 98 or later Windows 98 Second Edition but maybe my hardware was different ??
I do not have any ‘mine’ screenshots from that period as I lost all my 40 GB (huge then) drive of data when I moved/resized the partition with Partition Magic to get some more space from the less filled C: drive. That day I learned hard that “there are people who do backups and people who will do backups”. I never lost data again as I had multiple copies of my data, but the same as Netheril fall the lost data was was gone forever.
I always followed various alternatives which led me to try Linux in 2003, after reading about various distributions philosophies I decided to run Slackware Linux with KDE 3. My buddy used Aurox Linux by then (one of the few Linux distributions from Poland) and encouraged me to do the same – especially in the context of fixing possible problems as he already knew it and also as he recently dumped Windows system. But Slackware sounded like a better idea so I took that path instead. At first I dual booted between Windows XP and Slackware Linux cause I had everything worked out on the Windows world while I often felt helpless in the Linux world, so I would reboot into Windows to play some games or find a solution for Linux problem if that was required. I remember how strange the concept of dual clipboards (PRIMARY and SECONDARY) was for me by then. I was amazed why ‘so much better’ system as Linux (at least marketed that way) needs a system tray program to literally manage the clipboard. On Windows it was obvious, you do [CTRL]+[C] to copy and [CTRL]+[V] to paste things, but on Linux there (no I know its X11 feature) there were two clipboards that were synchronized by this little system tray program from KDE 3. It was also unthinkable for me that I will ‘lost’ contents of last/recent [CTRL]+[C] operation if I close the application from which the copy was made. I settled down a little on Slackware but not for long. I really did not liked manual dependency management for packages for example. Also KDE 3 was really ugly and despite trying all possible options I was not able to tweak it into something nice looking.
After half a year on Slackware I checked the Linux distributions again and decided to try Gentoo Linux. I definitely agree with the image below which visualizes Gentoo Linux experience, especially when You install it for he first time ??
Of course I went with the most hardcore version with self building Stage 1 (compiler and toolchain) which was horrible idea at that time because compilation on slow single core machine took forever … but after many hours I got Gentoo installed. I now have to decide which desktop environment to use. I have read a lot of good news about Fluxbox at that time so this is what I tried. It was very weird experience (to create everything in GUI from scratch) but very pleasant one. That recalled me the times of AMIGA … but Linux came in the way too much often. The more I dig into Gentoo Linux the more I read that lots of Gentoo features are based on FreeBSD solutions. Gentoo Portage is a clone of FreeBSD Ports. That ‘central’ /etc/rc.conf system configuration file concept was taken from FreeBSD as well. So I started to gather information about FreeBSD. The (then) FreeBSD website or FreeBSD Ports site (still) felt little outdated to say the least but that did not discouraged me.
Somewhere in 2005 I installed FreeBSD 5.4 on my computer. The beginnings were hard, like the earlier step with Gentoo but similarly like Gentoo the FreeBSD project came with a lot of great documentation. While Gentoo documentation is concentrated within various Gentoo Wiki sites the FreeBSD project comes with ‘official’ documentation in the form of Handbook and FAQ. I remember my first questions at the now nonexistent BSDForums.org site – for example one of the first ones – how to scroll the terminal output in the plain console. I now know that I had to push Scroll Lock button but it was something totally new for me.
Why FreeBSD and not OpenBSD or NetBSD? Probably because Gentoo based most their concepts on the FreeBSD solutions, so that led me to FreeBSD instead of the other BSD operating systems. Currently I still use FreeBSD but I keep an steady eye on the OpenBSD, HardenedBSD and DragonFly BSD solutions and improvements.
As the migration path from Linux to FreeBSD is a lot easier – all configuration files from /home can be just copied – the migration was quite fast easy. I again had the Fluxbox configuration which I used on the Gentoo. Now – on FreeBSD – it started to fell even more like AMIGA times. Everything is/has been well thought and had its place and reason. The documentation was good and the FreeBSD Community was second to none.
After 15 years of using various Windows, UNIX (macOS/AIX/HP-UX/Solaris/OpenSolaris/Illumos/FreeBSD/OpenBSD/NetBSD) and UNIX-like (Linux) systems I always come to conclusion that FreeBSD is the system that sucks least. And sucks least with each release and one day I will write why FreeBSD is such great operating system … if I already haven’t

##News Roundup
OpenBSD on the Desktop: some thoughts

I’ve been using OpenBSD on my ThinkPad X230 for some weeks now, and the experience has been peculiar in some ways.
The OS itself in my opinion is not ready for widespread desktop usage, and the development team is not trying to push it in the throat of anybody who wants a Windows or macOS alternative. You need to understand a little bit of how *NIX systems work, because you’ll use CLI more than UI. That’s not necessarily bad, and I’m sure I learned a trick or two that could translate easily to Linux or macOS. Their development process is purely based on developers that love to contribute and hack around, just because it’s fun. Even the mailing list is a cool place to hang on! Code correctness and security are a must, nothing gets committed if it doesn’t get reviewed thoroughly first - nowadays the first two properties should be enforced in every major operating system.
I like the idea of a platform that continually evolves. pledge(2) and unveil(2) are the proof that with a little effort, you can secure existing software better than ever.
I like the “sensible defaults” approach, having an OS ready to be used - UI included if you selected it during the setup process - is great.
Just install a browser and you’re ready to go.
Manual pages on OpenBSD are real manuals, not an extension of the “–help” command found in most CLI softwares. They help you understand inner workings of the operating system, no internet connection needed. There are some trade-offs, too.
Performance is not first-class, mostly because of all the security mitigations and checks done at runtime.
I write Go code in neovim, and sometimes you can feel a slight slowdown when you’re compiling and editing multiple files at the same time, but usually I can’t notice any meaningful difference. Browsers are a different matter though, you can definitely feel something differs from the experience you can have on mainstream operating systems. But again, trade-offs.
To use OpenBSD on the desktop you must be ready to sacrifice some of the goodies of mainstream OSes, but if you’re searching for a zen place to do your computing stuff, it’s the best you can get right now.

###The history of file type information being available in Unix directories

The two things that Unix directory entries absolutely have to have are the name of the directory entry and its ‘inode’, by which we generically mean some stable kernel identifier for the file that will persist if it gets renamed, linked to other directories, and so on. Unsurprisingly, directory entries have had these since the days when you read the raw bytes of directories with read(), and for a long time that was all they had; if you wanted more than the name and the inode number, you had to stat() the file, not just read the directory. Then, well, I’ll quote myself from an old entry on a find optimization:
[…], Unix filesystem developers realized that it was very common for programs reading directories to need to know a bit more about directory entries than just their names, especially their file types (find is the obvious case, but also consider things like ‘ls -F’). Given that the type of an active inode never changes, it’s possible to embed this information straight in the directory entry and then return this to user level, and that’s what developers did; on some systems, readdir(3) will now return directory entries with an additional dtype field that has the directory entry’s type.
On Twitter, I recently grumbled about Illumos not having this dtype field. The ensuing conversation wound up with me curious about exactly where dtype came from and how far back it went. The answer turns out to be a bit surprising due to there being two sides of dtype.
On the kernel side, dtype appears to have shown up in 4.4 BSD. The 4.4 BSD /usr/src/sys/dirent.h has a struct dirent that has a dtype field, but the field isn’t documented in either the comments in the file or in the getdirentries(2) manpage; both of those admit only to the traditional BSD dirent fields. This 4.4 BSD dtype was carried through to things that inherited from 4.4 BSD (Lite), specifically FreeBSD, but it continued to be undocumented for at least a while.
(In FreeBSD, the most convenient history I can find is here, and the dtype field is present in sys/dirent.h as far back as FreeBSD 2.0, which seems to be as far as the repo goes for releases.)
Documentation for dtype appeared in the getdirentries(2) manpage in FreeBSD 2.2.0, where the manpage itself claims to have been updated on May 3rd 1995 (cf). In FreeBSD, this appears to have been part of merging 4.4 BSD ‘Lite2’, which seems to have been done in 1997. I stumbled over a repo of UCB BSD commit history, and in it the documentation appears in this May 3rd 1995 change, which at least has the same date. It appears that FreeBSD 2.2.0 was released some time in 1997, which is when this would have appeared in an official release.
In Linux, it seems that a dirent structure with a dtype member appeared only just before 2.4.0, which was released at the start of 2001. Linux took this long because the dtype field only appeared in the 64-bit ‘large file support’ version of the dirent structure, and so was only return by the new 64-bit getdents64() system call. This would have been a few years after FreeBSD officially documented dtype, and probably many years after it was actually available if you peeked at the structure definition.
As far as I can tell, dtype is present on Linux, FreeBSD, OpenBSD, NetBSD, Dragonfly BSD, and Darwin (aka MacOS or OS X). It’s not present on Solaris and thus Illumos. As far as other commercial Unixes go, you’re on your own; all the links to manpages for things like AIX from my old entry on the remaining Unixes appear to have rotted away.
Sidebar: The filesystem also matters on modern Unixes
Even if your Unix supports dtype in directory entries, it doesn’t mean that it’s supported by the filesystem of any specific directory. As far as I know, every Unix with dtype support has support for it in their normal local filesystems, but it’s not guaranteed to be in all filesystems, especially non-Unix ones like FAT32. Your code should always be prepared to deal with a file type of DTUNKNOWN.
It’s also possible to have things the other way around, where you have a filesystem with support for file type information in directories that’s on a Unix that doesn’t support it. There are a number of plausible reasons for this to happen, but they’re either obvious or beyond the scope of this entry.

###Multiboot Pinebook KDE neon

Recently a KDE neon image for the Pinebook was announced. There is a new image, with a handful of fixes, which the KDE Plasma team has been working on over the past week and a half.
Here’s a picture of my Pinebook running KDE neon — watching Panic! At the Disco’s High Hopes — sitting in front of my monitor that’s hooked up to one of my openSUSE systems. There are still some errata, and watching video sucks up battery, but for hacking on documentation from my hammock in the garden, or doing IRC meetings it’s a really nice machine.
But one of the neat things about running KDE neon off of an SD card on the Pinebook is that it’s portable — that SD card can move around. So let’s talk about multiboot in the sense of “booting the same OS storage medium in different hardware units” rather than “booting different OS from a medium in a single hardware unit”. On these little ARM boards, u-boot does all the heavy lifting early in the boot process. So to re-use the KDE neon Pinebook image on another ARM board, the u-boot blocks need to be replaced.
I have the u-boot from a Pine64 image (I forget what) lying around, 1015 blocks of 1024 bytes, which I can dd over the u-boot blocks on the SD card, dd bs=1k conv=notrunc,sync if=uboot.img of=/dev/da0 seek=8, and then the same SD card, with the filesystem and data from the Pinebook, will boot on the Pine64 board. Of course, to move the SD card back again, I need to restore the Pinebook u-boot blocks.
Here’s a picture of my Pineboard (the base is a piece of the garden fence, it’s Douglas pine, with 4mm threaded rods acting as the corner posts for my Pine64 mini-rack), with power and network and a serial console attached, along with the serial console output of the same.
The nice thing here is that the same software stack runs on the Pine64 but then has a wired network — which in turn means that if I switch on the other boards in that mini-rack, I’ve got a distcc-capable cluster for fast development, and vast NFS storage (served from ZFS on my FreeBSD machines) for source. I can develop in a high(er) powered environment, and then swap the card around into the Pinebook for testing-on-the-go.
So to sum up: you can multiboot the KDE neon Pinebook image on other Pine64 hardware (i.e. the Pine64 board). To do so, you need to swap around u-boot blocks. The blocks can be picked out of an image built for each board, and then a particular image (e.g. the latest KDE neon Pinebook) can be run on either board.

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Wed, 03 Oct 2018 10:00:00 -0700
Episode 265: Software Disenchantment | BSD Now 265
We report from our experiences at EuroBSDcon, disenchant software, LLVM 7.0.0 has been released, Thinkpad BIOS update options, HardenedBSD Foundation announced, and ZFS send vs. rsync.

##Headlines

###[FreeBSD DevSummit & EuroBSDcon 2018 in Romania]

  • Your hosts are back from EuroBSDcon 2018 held in Bucharest, Romania this year. The first two days of the conference are used for tutorials and devsummits (FreeBSD and NetBSD), while the last two are for talks.
  • Although Benedict organized the devsummit in large parts, he did not attend it this year. He held his Ansible tutorial in the morning of the first day, followed by Niclas Zeising’s new ports and poudriere tutorial (which had a record attendance). It was intended for beginners that had never used poudriere before and those who wanted to create their first port. The tutorial was well received and Niclas already has ideas for extending it for future conferences.
  • On the second day, Benedict took Kirk McKusick’s “An Introduction to the FreeBSD Open-Source Operating System” tutorial, held as a one full day class this year. Although it was reduced in content, it went into enough depth of many areas of the kernel and operating system to spark many questions from attendees. Clearly, this is a good start into kernel programming as Kirk provides enough material and backstories to understand why certain things are implemented as they are.
  • Olivier Robert took https://www.talegraph.com/tales/l2o9ltrvsE (pictures from the devsummit) and created a nice gallery out of it.
  • Devsummit evenings saw dinners at two restaurants that allowed developers to spend some time talking over food and drinks.
  • The conference opened on the next day with the opening session held by Mihai Carabas. He introduced the first keynote speaker, a colleague of his who presented “Lightweight virtualization with LightVM and Unikraft”.
  • Benedict helped out at the FreeBSD Foundation sponsor table and talked to people. He saw the following talks in between:

Selfhosting as an alternative to the public cloud (by Albert Dengg)
Using Boot Environments at Scale (by Allan Jude)
Livepatching FreeBSD kernel (by Maciej Grochowski)
FreeBSD: What to (Not) Monitor (by Andrew Fengler)
FreeBSD Graphics (by Niclas Zeising)

  • Allan spent a lot of time talking to people and helping track down issues they were having, in addition to attending many talks:

    Hacking together a FreeBSD presentation streaming box – For as little as possible (by Tom Jones)
    Introduction of FreeBSD in new environments (by Baptiste Daroussin)
    Keynote: Some computing and networking historical perspectives (by Ron Broersma)
    Livepatching FreeBSD kernel (by Maciej Grochowski)
    FreeBSD: What to (Not) Monitor (by Andrew Fengler)
    Being a BSD user (by Roller Angel)
    From “Hello World” to the VFS Layer: building a beadm for DragonFly BSD (by Michael Voight)

  • We also met the winner of our Power Bagel raffle from Episode 2^8. He received the item in the meantime and had it with him at the conference, providing a power outlet to charge other people’s devices.
  • During the closing session, GroffTheBSDGoat was handed over to Deb Goodkin, who will bring the little guy to the Grace Hopper Celebration of Women in Computing conference and then to MeetBSD later this year. It was also revealed that next year’s EuroBSDcon will be held in Lillehammer, Norway.
  • Thanks to all the speakers, helpers, sponsors, organizers, and attendees for making it a successful conferences. There were no talks recorded this year, but the slides will be uploaded to the EuroBSDcon website in a couple of weeks. The OpenBSD talks are already available, so check them out.

###Software disenchantment

I’ve been programming for 15 years now. Recently our industry’s lack of care for efficiency, simplicity, and excellence started really getting to me, to the point of me getting depressed by my own career and the IT in general.
Modern cars work, let’s say for the sake of argument, at 98% of what’s physically possible with the current engine design. Modern buildings use just enough material to fulfill their function and stay safe under the given conditions. All planes converged to the optimal size/form/load and basically look the same.
Only in software, it’s fine if a program runs at 1% or even 0.01% of the possible performance. Everybody just seems to be ok with it. People are often even proud about how much inefficient it is, as in “why should we worry, computers are fast enough”:
@tveastman: I have a Python program I run every day, it takes 1.5 seconds. I spent six hours re-writing it in rust, now it takes 0.06 seconds. That efficiency improvement means I’ll make my time back in 41 years, 24 days :-)
You’ve probably heard this mantra: “programmer time is more expensive than computer time”. What it means basically is that we’re wasting computers at an unprecedented scale. Would you buy a car if it eats 100 liters per 100 kilometers? How about 1000 liters? With computers, we do that all the time.

  • Everything is unbearably slow

Look around: our portable computers are thousands of times more powerful than the ones that brought man to the moon. Yet every other webpage struggles to maintain a smooth 60fps scroll on the latest top-of-the-line MacBook Pro. I can comfortably play games, watch 4K videos but not scroll web pages? How is it ok?
Google Inbox, a web app written by Google, running in Chrome browser also by Google, takes 13 seconds to open moderately-sized emails:
It also animates empty white boxes instead of showing their content because it’s the only way anything can be animated on a webpage with decent performance. No, decent doesn’t mean 60fps, it’s rather “as fast as this web page could possibly go”. I’m dying to see web community answer when 120Hz displays become mainstream. Shit barely hits 60Hz already.
Windows 10 takes 30 minutes to update. What could it possibly be doing for that long? That much time is enough to fully format my SSD drive, download a fresh build and install it like 5 times in a row.
Pavel Fatin: Typing in editor is a relatively simple process, so even 286 PCs were able to provide a rather fluid typing experience.
Modern text editors have higher latency than 42-year-old Emacs. Text editors! What can be simpler? On each keystroke, all you have to do is update tiny rectangular region and modern text editors can’t do that in 16ms. It’s a lot of time. A LOT. A 3D game can fill the whole screen with hundreds of thousands (!!!) of polygons in the same 16ms and also process input, recalculate the world and dynamically load/unload resources. How come?
As a general trend, we’re not getting faster software with more features. We’re getting faster hardware that runs slower software with the same features. Everything works way below the possible speed. Ever wonder why your phone needs 30 to 60 seconds to boot? Why can’t it boot, say, in one second? There are no physical limitations to that. I would love to see that. I would love to see limits reached and explored, utilizing every last bit of performance we can get for something meaningful in a meaningful way.

  • Everything is HUUUUGE

And then there’s bloat. Web apps could open up to 10× faster if you just simply block all ads. Google begs everyone to stop shooting themselves in their feet with AMP initiative—a technology solution to a problem that doesn’t need any technology, just a little bit of common sense. If you remove bloat, the web becomes crazy fast. How smart do you have to be to understand that?
Android system with no apps takes almost 6 Gb. Just think for a second how obscenely HUGE that number is. What’s in there, HD movies? I guess it’s basically code: kernel, drivers. Some string and resources too, sure, but those can’t be big. So, how many drivers do you need for a phone?
Windows 95 was 30Mb. Today we have web pages heavier than that! Windows 10 is 4Gb, which is 133 times as big. But is it 133 times as superior? I mean, functionally they are basically the same. Yes, we have Cortana, but I doubt it takes 3970 Mb. But whatever Windows 10 is, is Android really 150% of that?
Google keyboard app routinely eats 150 Mb. Is an app that draws 30 keys on a screen really five times more complex than the whole Windows 95? Google app, which is basically just a package for Google Web Search, is 350 Mb! Google Play Services, which I do not use (I don’t buy books, music or videos there)—300 Mb that just sit there and which I’m unable to delete.
All that leaves me around 1 Gb for my photos after I install all the essential (social, chats, maps, taxi, banks etc) apps. And that’s with no games and no music at all! Remember times when an OS, apps and all your data fit on a floppy?
Your desktop todo app is probably written in Electron and thus has userland driver for Xbox 360 controller in it, can render 3d graphics and play audio and take photos with your web camera.
A simple text chat is notorious for its load speed and memory consumption. Yes, you really have to count Slack in as a resource-heavy application. I mean, chatroom and barebones text editor, those are supposed to be two of the less demanding apps in the whole world. Welcome to 2018.
At least it works, you might say. Well, bigger doesn’t imply better. Bigger means someone has lost control. Bigger means we don’t know what’s going on. Bigger means complexity tax, performance tax, reliability tax. This is not the norm and should not become the norm. Overweight apps should mean a red flag. They should mean run away scared.

  • Better world manifesto

I want to see progress. I want change. I want state-of-the-art in software engineering to improve, not just stand still. I don’t want to reinvent the same stuff over and over, less performant and more bloated each time. I want something to believe in, a worthy end goal, a future better than what we have today, and I want a community of engineers who share that vision.
What we have today is not progress. We barely meet business goals with poor tools applied over the top. We’re stuck in local optima and nobody wants to move out. It’s not even a good place, it’s bloated and inefficient. We just somehow got used to it.
So I want to call it out: where we are today is bullshit. As engineers, we can, and should, and will do better. We can have better tools, we can build better apps, faster, more predictable, more reliable, using fewer resources (orders of magnitude fewer!). We need to understand deeply what are we doing and why. We need to deliver: reliably, predictably, with topmost quality. We can—and should–take pride in our work. Not just “given what we had…”—no buts!
I hope I’m not alone at this. I hope there are people out there who want to do the same. I’d appreciate if we at least start talking about how absurdly bad our current situation in the software industry is. And then we maybe figure out how to get out.

##News Roundup
[llvm-announce] LLVM 7.0.0 Release

I am pleased to announce that LLVM 7 is now available. Get it here: https://llvm.org/releases/download.html#7.0.0 The release contains the work on trunk up to SVN revision 338536 plus work on the release branch. It is the result of the community's work over the past six months, including: function multiversioning in Clang with the 'target' attribute for ELF-based x86/x86_64 targets, improved PCH support in clang-cl, preliminary DWARF v5 support, basic support for OpenMP 4.5 offloading to NVPTX, OpenCL C++ support, MSan, X-Ray and libFuzzer support for FreeBSD, early UBSan, X-Ray and libFuzzer support for OpenBSD, UBSan checks for implicit conversions, many long-tail compatibility issues fixed in lld which is now production ready for ELF, COFF and MinGW, new tools llvm-exegesis, llvm-mca and diagtool. And as usual, many optimizations, improved diagnostics, and bug fixes. For more details, see the release notes: https://llvm.org/releases/7.0.0/docs/ReleaseNotes.html https://llvm.org/releases/7.0.0/tools/clang/docs/ReleaseNotes.html https://llvm.org/releases/7.0.0/tools/clang/tools/extra/docs/ReleaseNotes.html https://llvm.org/releases/7.0.0/tools/lld/docs/ReleaseNotes.html Thanks to everyone who helped with filing, fixing, and code reviewing for the release-blocking bugs! Special thanks to the release testers and packagers: Bero Rosenkränzer, Brian Cain, Dimitry Andric, Jonas Hahnfeld, Lei Huang Michał Górny, Sylvestre Ledru, Takumi Nakamura, and Vedant Kumar. For questions or comments about the release, please contact the community on the mailing lists. Onwards to LLVM 8! Cheers, Hans

###Update your Thinkpad’s bios with Linux or OpenBSD

  • Get your new bios

At first, go to the Lenovo website and download your new bios:

  • Go to lenovo support
  • Use the search bar to find your product (example for me, x270)
  • Choose the right product (if necessary) and click search
  • On the right side, click on Update Your System
  • Click on BIOS/UEFI
  • Choose *BIOS Update (Bootable CD) for Windows *
  • Download

For me the file is called like this : r0iuj25wd.iso

  • Extract bios update

Now you will need to install geteltorito.

  • With OpenBSD:

$ doas pkgadd geteltorito
quirks-3.7 signed on 2018-09-09T13:15:19Z
geteltorito-0.6: ok

  • With Debian:

$ sudo apt-get install genisoimage

  • Now we will extract the bios update :

$ geteltorito -o biosupdate.img r0iuj25wd.iso
Booting catalog starts at sector: 20
Manufacturer of CD: NERO BURNING ROM VER 12
Image architecture: x86
Boot media type is: harddisk
El Torito image starts at sector 27 and has 43008 sector(s) of 512 Bytes

Image has been written to file "biosupdate.img".
This will create a file called biosupdate.img.

  • Put the image on an USB key
  • CAREFULL : on my computer, my USB key is sda1 on Linux and sd1 on OpenBSD.

Please check twice on your computer the name of your USB key.

  • With OpenBSD :

$ doas dd if=biosupdate.img of=/dev/rsd1c

  • With Linux :

$ sudo dd if=biosupdate.img of=/dev/sda

Now all you need is to reboot, to boot on your USB key and follow the instructions. Enjoy 😉

###Announcing The HardenedBSD Foundation

In June of 2018, we announced our intent to become a not-for-profit, tax-exempt 501©(3) organization in the United States. It took a dedicated team months of work behind-the-scenes to make that happen. On 06 September 2018, HardenedBSD Foundation Corp was granted 501©(3) status, from which point all US-based persons making donations can deduct the donation from their taxes.
We are grateful for those who contribute to HardenedBSD in whatever way they can. Thank you for making HardenedBSD possible. We look forward to a bright future, driven by a helpful and positive community.

###How you migrate ZFS filesystems matters

If you want to move a ZFS filesystem around from one host to another, you have two general approaches; you can use ‘zfs send’ and ‘zfs receive’, or you can use a user level copying tool such as rsync (or ‘tar -cf | tar -xf’, or any number of similar options). Until recently, I had considered these two approaches to be more or less equivalent apart from their convenience and speed (which generally tilted in favour of ‘zfs send’). It turns out that this is not necessarily the case and there are situations where you will want one instead of the other.
We have had two generations of ZFS fileservers so far, the Solaris ones and the OmniOS ones. When we moved from the first generation to the second generation, we migrated filesystems across using ‘zfs send’, including the filesystem with my home directory in it (we did this for various reasons). Recently I discovered that some old things in my filesystem didn’t have file type information in their directory entries. ZFS has been adding file type information to directories for a long time, but not quite as long as my home directory has been on ZFS.
This illustrates an important difference between the ‘zfs send’ approach and the rsync approach, which is that zfs send doesn’t update or change at least some ZFS on-disk data structures, in the way that re-writing them from scratch from user level does. There are both positives and negatives to this, and a certain amount of rewriting does happen even in the ‘zfs send’ case (for example, all of the block pointers get changed, and ZFS will re-compress your data as applicable).
I knew that in theory you had to copy things at the user level if you wanted to make sure that your ZFS filesystem and everything in it was fully up to date with the latest ZFS features. But I didn’t expect to hit a situation where it mattered in practice until, well, I did. Now I suspect that old files on our old filesystems may be partially missing a number of things, and I’m wondering how much of the various changes in ‘zfs upgrade -v’ apply even to old data.
(I’d run into this sort of general thing before when I looked into ext3 to ext4 conversion on Linux.)
With all that said, I doubt this will change our plans for migrating our ZFS filesystems in the future (to our third generation fileservers). ZFS sending and receiving is just too convenient, too fast and too reliable to give up. Rsync isn’t bad, but it’s not the same, and so we only use it when we have to (when we’re moving only some of the people in a filesystem instead of all of them, for example).
PS: I was going to try to say something about what ‘zfs send’ did and didn’t update, but having looked briefly at the code I’ve concluded that I need to do more research before running my keyboard off. In the mean time, you can read the OpenZFS wiki page on ZFS send and receive, which has plenty of juicy technical details.
PPS: Since eliminating all-zero blocks is a form of compression, you can turn zero-filled files into sparse files through a ZFS send/receive if the destination has compression enabled. As far as I know, genuine sparse files on the source will stay sparse through a ZFS send/receive even if they’re sent to a destination with compression off.

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 27 Sep 2018 01:00:00 -0700
Episode 264: Optimized-out | BSD Now 264
FreeBSD and DragonflyBSD benchmarks on AMD’s Threadripper, NetBSD 7.2 has been released, optimized out DTrace kernel symbols, stuck UEFI bootloaders, why ed is not a good editor today, tell your BSD story, and more.

##Headlines
FreeBSD & DragonFlyBSD Put Up A Strong Fight On AMD’s Threadripper 2990WX, Benchmarks Against Linux

The past two weeks I have been delivering a great deal of AMD Threadripper 2990WX benchmarks on Linux as well as some against Windows and Windows Server. But recently I got around to trying out some of the BSD operating systems on this 32-core / 64-thread processor to see how they would run and to see whether they would have similar scaling issues or not like we’ve seen on the Windows side against Linux. In this article are FreeBSD and DragonFlyBSD benchmarks with the X399 + 2990WX compared to a few Linux distributions.
The BSDs I focused my testing on were FreeBSD 11.2-STABLE and 12.0-CURRENT/ALPHA1 (the version in development) as well as iX System’s TrueOS that is tracking FreeBSD 12.0-CURRENT. Also included were DragonFlyBSD, with FreeBSD and DragonFlyBSD being tied as my favorite operating systems when it comes to the BSDs. When it came to FreeBSD 11.2-STABLE and 12.0-ALPHA1 on the Threadripper 2990WX, it worked out surprisingly well. I encountered no real issues during my two days of benchmarking on FreeBSD (and TrueOS). It was a great experience and FreeBSD was happy to exploit the 64 threads on the system.
DragonFlyBSD was a bit of a different story… Last week when I started this BSD testing I tried DragonFly 5.2.2 as the latest stable release as well as a DragonFlyBSD 5.3 development snapshot from last week: both failed to boot in either BIOS or UEFI modes.
But then a few days ago DragonFlyBSD lead developer Matthew Dillon bought himself a 2990WX platform. He made the necessary changes to get DragonFlyBSD 5.3 working and he ended up finding really great performance and potential out of the platform. So I tried the latest DragonFlyBSD 5.3 daily ISO on 22 August and indeed it now booted successfully and we were off to the races. Thus there are some DragonFlyBSD 5.3 benchmarks included in this article too.
Just hours ago, Matthew Dillon landed some 2990WX topology and scheduler enhancements but that fell out of the scope of when DragonFly was installed on this system. But over the weekend or so I plan to re-test DragonFlyBSD 5.3 and see how those optimizations affect the overall 2990WX performance now on that BSD. DragonFlyBSD 5.4 stable should certainly be an interesting release on several fronts!
With FreeBSD 11.2-STABLE and 12.0-ALPHA1 I ran benchmarks when using their stock compiler (LLVM Clang 6.0) as well as GCC 7.3 obtained via GCC 7.3. That was done to rule out compiler differences in benchmarking against the GCC-based Linux distributions. On DragonFlyBSD 5.3 it defaults to the GCC 5.4.1 but via pkg I also did a secondary run when upgraded to GCC 7.3.
The hardware and BIOS/UEFI settings were maintained the same throughout the entire benchmarking process. The system was made up of the AMD Ryzen Threadripper 2990WX at stock speeds, the ASUS ROG ZENITH EXTREME motherboard, 4 x 8GB DDR4-3200MHz memory, Samsung 970 EVO 500GB NVMe SSD, and Radeon RX Vega 56 graphics card.
All of these Linux vs. BSD benchmarks were carried out in a fully-automated and reproducible manner using the open-source Phoronix Test Suite benchmarking framework.
While for the last of today’s BSD vs. Linux benchmarking on the Threadripper 2990WX, the Linux distributions came out slightly ahead of FreeBSD and DragonFlyBSD with GCC (another test having issues with Clang 6.0 on the BSDs).
Overall, I was quite taken away by the BSD performance on the Threadripper 2990WX – particularly FreeBSD. In a surprising number of benchmarks, the BSDs were outperforming the tested Linux distributions though often by incredibly thin margins. Still, quite an accomplishment for these BSD operating systems and considering how much better Linux is already doing than Windows 10 / Windows Server on this 32-core / 64-thread processor. Then again, the BSDs like Linux have a long history of running on high core/thread-count systems, super computers, and other HPC environments.
It will be interesting to see how much faster DragonFlyBSD can run given today’s commit to its kernel with scheduler and topology improvements for the 2990WX. Those additional DragonFlyBSD benchmarks will be published in the coming days once they are completed.

###NetBSD 7.2 released

The NetBSD Project is pleased to announce NetBSD 7.2, the second feature update of the NetBSD 7 release branch. It represents a selected subset of fixes deemed important for security or stability reasons, as well as new features and enhancements.

  • General Security Note

The NetBSD 7.2 release is a maintenance release of the netbsd-7 branch, which had it's first major release, NetBSD 7.0 in September 2015. A lot of security features have been added to later NetBSD versions, and for new installations we highly recommend using our latest release, NetBSD 8.0 instead.

  • Some highlights of the 7.2 release are:
  • Support for USB 3.0.
  • Enhancements to the Linux emulation subsystem.
  • Fixes in binary compatibility for ancient NetBSD executables.
  • iwm(4) driver for Intel Wireless 726x, 316x, 826x and 416x series added.
  • Support for Raspberry Pi 3 added.
  • Fix interrupt setup on Hyper-V VMs with Legacy Network Adapter.
  • SVR4 and IBCS2 compatibility subsystems have been disabled by default (besides IBCS2 on VAX). These subsystems also do not auto-load their modules any more.
  • Various USB stability enhancements.
  • Numerous bug fixes and stability improvements.

Complete source and binaries for NetBSD 7.2 are available for download at many sites around the world. A list of download sites providing FTP, AnonCVS, SUP, and other services may be found at https://www.NetBSD.org/mirrors/. We encourage users who wish to install via ISO or USB disk images to download via BitTorrent by using the torrent files supplied in the images area. A list of hashes for the NetBSD 7.2 distribution has been signed with the well-connected PGP key for the NetBSD Security Officer: https://cdn.NetBSD.org/pub/NetBSD/security/hashes/NetBSD-7.2_hashes.asc
NetBSD is free. All of the code is under non-restrictive licenses, and may be used without paying royalties to anyone. Free support services are available via our mailing lists and website. Commercial support is available from a variety of sources. More extensive information on NetBSD is available from our website:

##News Roundup
Including optimized-out kernel symbols in dtrace on FreeBSD

Have you ever had dtrace(1) on FreeBSD fail to list a probe that should exist in the kernel? This is because Clang will optimize-out some functions. The result is ctfconvert(1) will not generate debugging symbols that dtrace(1) uses to identify probes. I have a quick solution to getting those probes visible to dtrace(1).

In my case, I was trying to instrument on ieee80211_ioctl_get80211, whose sister function ieee80211_ioctl_set80211 has a dtrace(1) probe in the generic FreeBSD 11 and 12 kernels. Both functions are located in /usr/src/sys/net80211/ieee80211_ioctl.c.

My first attempt was to add to /etc/make.conf as follows and recompile the kernel.

CFLAGS+=-O0 and -fno-inline-functions

This failed to produce the dtrace(1) probe. Several other attempts failed and I was getting inconsistent compilation results (Is it me or is ieee80211_ioctl.c compiled with different flags if NO_CLEAN=1 is set?). When I manually compiled the object file by copying the compilation line for the object file and adding -O0 -fno-inline-functions, nm(1) on both the object file and kernel demonstrated that the symbol was present. I installed the kernel, rebooted and it was listed as a dtrace probe. Great!

But as I continued to debug my WiFi driver (oh yeah, I’m very slowly extending rtwn(4)), I found myself rebuilding the kernel several times and frequently rebooting. Why not do this across the entire kernel?

After hacking around, my solution was to modify the build scripts. My solution was to edit /usr/src/sys/conf/kern.pre.mk and modify all optimization level 2 to optimization level 0. The following is my diff(1) on FreeBSD 12.0-CURRENT.

  • A few thoughts:

This seems like a hack rather than a long-term solution. Either the problem is with the hard-coded optimization flags, or the inability to overwrite them in all places in make.conf.
Removing optimizations is only something I would do in a non-production kernel, so its as if I have to choose between optimizations for a production kernel or having dtrace probes. But dtrace explicitly markets itself as not impactful on production.
Using the dtrace pony as your featured image on WordPress does not render properly and must be rotated and modified. Blame Bryan Cantrill.
If you have a better solution, please let me know and I will update the article, but this works for me!

###FreeBSD: UEFI Bootloader stuck on BootCurrent/BootOrder/BootInfo on Asus Motherboards (and fix!)

Starting with FreeBSD CURRENT from about a few weeks of posting date, but including FreeBSD 12 alpha releases (not related to DEC Alpha), I noticed one thing: When I boot FreeBSD from UEFI on a homebuilt desktop with a Asus H87M-E motherboard, and have Root on ZFS, the bootloader gets stuck on lines like BootCurrent, BootOrder, and BootInfo. This issue occurs when I try to boot directly to efi\boot\bootx64.efi.

One person had a similar issue on a Asus H87I-PLUS motherboard. This issue may or may not exist on other Asus motherboards, desktops, or laptops. This may be specific to Asus motherboards for Intel’s Haswell, but may also exist on newer systems (e.g. Skylake) or older (e.g. Ivy Bridge) with Asus motherboards, as well as Asus desktops or laptops.

  • There are two solutions to this problem:
  • Use Legacy BIOS mode instead of UEFI mode
  • Install a FreeBSD UEFI Boot entry

Keep in mind that I am not going to talk about this issue and third-party UEFI boot managers such as rEFInd here.
The first option is rather straightforward: you need to make sure your computer has “Secure Boot” disabled and “Legacy Boot” or “CSM” enabled. Then, you need to make sure FreeBSD is installed in BIOS mode. However, this solution is (in my opinion) suboptimal. Why? Because:
You won’t be able to use hard drives bigger than 2TB
You are limited to MBR Partitioning on Asus motherboards with UEFI as Asus motherboards refuse to boot GPT partitioned disks in BIOS mode
Legacy BIOS mode may not exist on future computers or motherboards (although those systems may not have this issue, and this issue may get fixed by then)
The second option, however, is less straightforward, but will let you keep UEFI. Many UEFI systems, including affected Asus motherboards described here, include a boot manager built into the UEFI. FreeBSD includes a tool called efibootmgr to manage this, similar to the similarly-named tool in Linux, but with a different syntax.

###Why ed(1) is not a good editor today

I’ll start with my tweet:

Heretical Unix opinion time: ed(1) may be the 'standard Unix editor', but it is not a particularly good editor outside of a limited environment that almost never applies today.

There is a certain portion of Unixdom that really likes ed(1), the ‘standard Unix editor’. Having actually used ed for a not insignificant amount of time (although it was the friendlier ‘UofT ed’ variant), I have some reactions to what I feel is sometimes overzealous praise of it. One of these is what I tweeted.
The fundamental limitation of ed is that it is what I call an indirect manipulation interface, in contrast to the explicit manipulation interfaces of screen editors like vi and graphical editors like sam (which are generally lumped together as ‘visual’ editors, so called because they actually show you the text you’re editing). When you edit text in ed, you have some problems that you don’t have in visual editors; you have to maintain in your head the context of what the text looks like (and where you are in it), you have to figure out how to address portions of that text in order to modify them, and finally you have to think about how your edit commands will change the context. Copious use of ed’s p command can help with the first problem, but nothing really deals with the other two. In order to use ed, you basically have to simulate parts of ed in your head.
Ed is a great editor in situations where the editor explicitly presenting this context is a very expensive or outright impossible operation. Ed works great on real teletypes, for example, or over extremely slow links where you want to send and receive as little data as possible (and on real teletypes you have some amount of context in the form of an actual printout that you can look back at). Back in the old days of Unix, this described a fairly large number of situations; you had actual teletypes, you had slow dialup links (and later slow, high latency network links), and you had slow and heavily overloaded systems.
However, that’s no longer the situation today (at least almost all of the time). Modern systems and links can easily support visual editors that continually show you the context of the text and generally let you more or less directly manipulate it (whether that is through cursoring around it or using a mouse). Such editors are easier and faster to use, and they leave you with more brainpower free to think about things like the program you’re writing (which is the important thing).
If you can use a visual editor, ed is not a particularly good editor to use instead; you will probably spend a lot of effort (and some amount of time) on doing by hand something that the visual editor will do for you. If you are very practiced at ed, maybe this partly goes away, but I maintain that you are still working harder than you need to be.
The people who say that ed is a quite powerful editor are correct; ed is quite capable (although sadly limited by only editing a single file). It’s just that it’s also a pain to use.
(They’re also correct that ed is the foundation of many other things in Unix, including sed and vi. But that doesn’t mean that the best way to learn or understand those things is to learn and use ed.)
This doesn’t make ed a useless, vestigial thing on modern Unix, though. There are uses for ed in non-interactive editing, for example. But on modern Unix, ed is a specialized tool, much like dc. It’s worth knowing that ed is there and roughly what it can do, but it’s probably not worth learning how to use it before you need it. And you’re unlikely to ever be in a situation where it’s the best choice for interactive editing (and if you are, something has generally gone wrong).
(But if you enjoy exploring the obscure corners of Unix, sure, go for it. Learn dc too, because it’s interesting in its own way and, like ed, it’s one of those classical old Unix programs.)

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Wed, 19 Sep 2018 22:00:00 -0700
Episode 263: Encrypt That Pool | BSD Now 263
Mitigating Spectre/Meltdown on HP Proliant servers, omniOS installation setup, debugging a memory corruption issue on OpenBSD, CfT for OpenZFS native encryption, Asigra TrueNAS backup appliance shown at VMworld, NetBSD 6 EoL, and more.

##Headlines
How to mitigate Spectre and Meltdown on an HP Proliant server with FreeBSD

As recently announced in a previous article I wanted to write a couple of guides on how to mitigate Spectre and Meltdown vulnerabilities in GNU/Linux and UNIX environments. It is always a good and I hope a standard practice to have your systems patched and if they aren’t for whatever the reason (that legacy thing you’re carrying on for ages) you may take the necessary extra steps to protect your environment. I never planned to do any article on patching anything. Nowadays it’s a no brainer and operating systems have provided the necessary tools for this to be easy and as smooth as possible. So why this article?
Spectre and Meltdown are both hardware vulnerabilities. Major ones. They are meaningful for several reasons among them the world wide impact since they affect Intel and AMD systems which are ubiquitous. And second because patching hardware is not as easy, for the manufacturer and for the users or administrators in charge of the systems. There is still no known exploit around left out in the open hitting servers or desktops anywhere. The question is not if it will ever happen. The question is when will it happen. And it may be sooner than later. This is why big companies, governments and people in charge of big deployments are patching or have already patched their systems. But have you done it to your system? I know you have a firewall. Have you thought about CVE-2018-3639? This particular one could make your browser being a vector to get into your system. So, no, there is no reason to skip this.
Patching these set of vulnerabilities implies some more steps and concerns than updating the operating system. If you are a regular Windows user I find rare you to be here and many of the things you will read may be foreign to you. I am not planning to do a guide on Windows systems since I believe someone else has or will do it and will do it better than me since I am not a pro Windows user. However there is one basic and common thing for all OS’s when dealing with Spectre and Meltdown and that is a microcode update is necessary for the OS patches to effectively work.
What is microcode? You can read the Wikipedia article but in short it is basically a layer of code that allows chip manufacturers to deal with modifications on the hardware they’ve produced and the operating systems that will manage that hardware. Since there’s been some issues (namely Spectre and Meltdown) Intel and AMD respectively have released a series of microcode updates to address those problems. First series did come with serious problems and some regressions, to the point GNU/Linux producers stopped releasing the microcode updates through their release channels for updates and placed the ball on Intel’s roof. Patching fast does always include risks, specially when dealing with hardware. OS vendors have resumed their microcode update releases so all seems to be fine now.
In order to update the microcode we’re faced with two options. Download the most recent BIOS release from our vendor, provided it patches the Spectre and Meltdown vulnerabilities, or patch it from the OS. If your hardware vendor has decided not to provide support on your hardware you are forced to use the latter solution. Yes, you can still keep your hardware. They usually come accompanied with a “release notes” file where there are some explanatory notes on what is fixed, what is new, etc. To make the search easy for you a news site collected the vendors list and linked the right support pages for anyone to look. In some scenarios it would be desirable not to replace the whole BIOS but just update the microcode from the OS side. In my case I should update an HP Proliant ML110 G7 box and the download link for that would be this.
Instead of using the full blown BIOS update path we’ll use the inner utilities to patch Spectre and Meltdown on FreeBSD. So let’s put our hands on it

  • See the article for the technical breakdown

###A look beyond the BSD teacup: OmniOS installation

Five years ago I wrote a post about taking a look beyond the Linux teacup. I was an Arch Linux user back then and since there were projects like ArchBSD (called PacBSD today) and Arch Hurd, I decided to take a look at and write about them. Things have changed. Today I’m a happy FreeBSD user, but it’s time again to take a look beyond the teacup of operating systems that I’m familiar with.

  • Why Illumos / OmniOS?

There are a couple of reasons. The Solaris derivatives are the other big community in the *nix family besides Linux and the BSDs and we hadn’t met so far. Working with ZFS on FreeBSD, I now and then I read messages that contain a reference to Illumos which certainly helps to keep up the awareness. Of course there has also been a bit of curiosity – what might the OS be like that grew ZFS?
Also the Ravenports project that I participate in planned to support Solaris/Illumos right from the beginning. I wanted to at least be somewhat “prepared” when support for that platform would finally land. So I did a little research on the various derivatives available and settled on the one that I had heard a talk about at last year’s conference of the German Unix Users Group: “OmniOS – Solaris for the Rest of Us”. I would have chosen SmartOS as I admire what Bryan Cantrill does but for getting to know Illumos I prefer a traditional installation over a run-from-RAM system.
Of course FreeBSD is not run by corporations, especially when compared to the state of Linux. And when it comes to sponsoring, OpenBSD also takes the money… When it comes to FreeBSD developers, there’s probably some truth to the claim that some of them are using macOS as their desktop systems while OpenBSD devs are more likely to develop on their OS of choice. But then there’s the statement that “every innovation in the past decade comes from Solaris”. Bhyve alone proves this wrong. But let’s be honest: Two of the major technologies that make FreeBSD a great platform today – ZFS and DTrace – actually do come from Solaris. PAM originates there and a more modern way of managing services as well. Also you hear good things about their zones and a lot of small utilities in general.
In the end it was a lack of time that made me cheat and go down the easiest road: Create a Vagrantfile and just pull a VM image of the net that someone else had prepared… This worked to just make sure that the Raven packages work on OmniOS. I was determined to return, though – someday. You know how things go: “someday” is a pretty common alias for “probably never, actually.”
But then I heard about a forum post on the BSDNow! podcast. The title “Initial OmniOS impressions by a BSD user” caught my attention. I read that it was written by somebody who had used FreeBSD for years but loathed the new Code of Conduct enough to leave. I also oppose the Conduct and have made that pretty clear in my February post [ ! -z ${COC} ] && exit 1. As stated there, I have stayed with my favorite OS and continue to advocate it. I decided to stop reading the post and try things out on my own instead. Now I’ve finally found the time to do so.

  • What’s next?

That’s it for part one. In part two I’ll try to make the system useful. So far I have run into a problem that I haven’t been able to solve. But I have some time now to figure things out for the next post. Let’s see if I manage to get it working or if I have to report failure!

###What are all these types of memory in top(1)?

  • Earlier this week I convinced Mark Johnston, one of the FreeBSD VM experts to update a page on the FreeBSD wiki that I saw was being referenced on stackoverflow and similar sites
  • Mark updated the explanations to be more correct, and to include more technical detail for inquiring minds
  • He also added the new type that appeared in FreeBSD somewhat recently

Active - Contains memory “actively” (recently) being used by applications
Inactive - Contains memory that has not been touched recently, or was released from the Buffer Cache
Laundry - Contains memory that Inactive but still potentially contains useful data that needs to be stored before this memory can be used again
Wired - Memory that cannot be swapped out, including the kernel, network stack, and the ZFS ARC
Buf - Buffer Cache, used my UFS and most filesystems except ZFS (which uses the ARC)
Free - Memory that is immediately available for use by the rest of the system

##News Roundup
OpenBSD saves me again! — Debug a memory corruption issue

Yesterday, I came across a third-part library issue, which crashes at allocating memory:

Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f594a5a9b6b in _int_malloc () from /usr/lib/libc.so.6
(gdb) bt
#0 0x00007f594a5a9b6b in _int_malloc () from /usr/lib/libc.so.6
#1 0x00007f594a5ab503 in malloc () from /usr/lib/libc.so.6
#2 0x00007f594b13f159 in operator new (sz=5767168) at /build/gcc/src/gcc/libstdc++-v3/libsupc++/new_op.cc:50

It is obvious that the memory tags are corrupted, but who is the murder? Since the library involves a lot of maths computation, it is not an easy task to grasp the code quickly. So I need to find another way:
(1) Open all warnings during compilation: -Wall. Nothing found.
(2) Use valgrind, but unfortunately, valgrind crashes itself:

valgrind: the 'impossible' happened:
Killed by fatal signal

host stacktrace:
==43326== at 0x58053139: get_bszB_as_is (m_mallocfree.c:303)
==43326== by 0x58053139: get_bszB (m_mallocfree.c:315)
==43326== by 0x58053139: vgPlain_arena_malloc (m_mallocfree.c:1799)
==43326== by 0x5800BA84: vgMemCheck_new_block (mc_malloc_wrappers.c:372)
==43326== by 0x5800BD39: vgMemCheck___builtin_vec_new (mc_malloc_wrappers.c:427)
==43326== by 0x5809F785: do_client_request (scheduler.c:1866)
==43326== by 0x5809F785: vgPlain_scheduler (scheduler.c:1433)
==43326== by 0x580AED50: thread_wrapper (syswrap-linux.c:103)
==43326== by 0x580AED50: run_a_thread_NORETURN (syswrap-linux.c:156)

sched status:
running_tid=1

(3) Change compiler, use clang instead of gcc, and hope it can give me some clues. Still no effect.
(4) Switch Operating System from Linux to OpenBSD, the program crashes again. But this time, it tells me where the memory corruption occurs:

Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x000014b07f01e52d in addMod (r=<error reading variable>, a=4693443247995522, b=28622907746665631,

I figure out the issue quickly, and not bother to understand the whole code. OpenBSD saves me again, thanks!

###Native Encryption for ZFS on FreeBSD (Call for Testing)

To anyone with an interest in native encryption in ZFS please test the projects/zfs-crypto-merge-0820 branch in my freebsd repo: https://github.com/mattmacy/networking.git

git clone https://github.com/mattmacy/networking.git -b projects/zfs-crypto-merge-0820

The UI is quite close to the Oracle Solaris ZFS crypto with minor differences for specifying key location.
Please note that once a feature is enabled on a pool it can’t be disabled. This means that if you enable encryption support on a pool you will never be able to import it in to a ZFS without encryption support. For this reason I would strongly advise against using this on any pool that can’t be easily replaced until this change has made its way in to HEAD after the freeze has been lifted.
By way of background the original ZoL commit can be found at:

###VMworld 2018: Showcasing Hybrid Cloud, Persistent Memory and the Asigra TrueNAS Backup Appliance

During its last year in Las Vegas before moving back to San Francisco, VMworld was abuzz with all the popular buzzwords, but the key focus was on supporting a more agile approach to hybrid cloud.
Surveys of IT stakeholders and analysts agree that most businesses have multiple clouds spanning both public cloud providers and private data centers. While the exact numbers vary, well over half of businesses have a hybrid cloud strategy consisting of at least three different clouds.
This focus on hybrid cloud provided the perfect timing for our announcement that iXsystems and Asigra are partnering to deliver the Asigra TrueNAS Backup Appliance, which combines Asigra Cloud Backup software backed by TrueNAS storage. Asigra TrueNAS Backup Appliances provide a self-healing and ransomware-resistent OpenZFS backup repository in your private cloud. The appliance can simultaneously be used as general-purpose file, block, and object storage. How does this tie in with the hybrid cloud? The Asigra Cloud Backup software can backup data from public cloud repositories – G Suite, Office 365, Salesforce, etc. – as well as intelligently move backed-up data to the public cloud for long-term retention.
Another major theme at the technical sessions was persistent memory, as vSphere 6.7 added support for persistent memory – either as a storage tier or virtualized and presented to a guest OS. As detailed in our blog post from SNIA’s Persistent Memory Summit 2018, persistent memory is rapidly becoming mainstream. Persistent memory bridges the gap between memory and flash storage – providing near-memory latency storage that persists across reboots or power loss. vSphere allows both legacy and persistent memory-aware applications to leverage this ultra-fast storage tier. We were excited to show off our newly-introduced TrueNAS M-Series at VMworld, as all TrueNAS M40 and M50 models leverage NVDIMM persistent memory technology to provide a super-fast write cache, or SLOG, without any of the limitations of Flash technology.
The iXsystems booth’s theme was “Enterprise Storage, Open Source Economics”. iXsystems leverages the power of Open Source software, combined with our enterprise-class hardware and support, to provide incredibly low TCO storage for virtualization environments. Our TrueNAS unified storage and server offerings are an ideal solution for your organization’s private cloud infrastructure. Combined with VMware NSX Hybrid Connect – formerly known as VMware Hybrid Cloud Extension – you can seamlessly shift running systems into a public cloud environment for a true hybrid cloud solution.
Another special treat at this year’s booth was iXsystems Vice President of Engineering Kris Moore giving demos of an early version of “Project TrueView”, a single-pane of glass management solution for administration of multiple FreeNAS and TrueNAS systems. In addition to simplified administration and enhanced monitoring, Project TrueView will also provide Role-Based Access Control for finer-grained permissions management. A beta version of Project TrueView is expected to be available at the end of this year.
Overall, we had a great week at VMworld 2018 with lots of good conversations with customers, press, analysts, and future customers about TrueNAS, the Asigra TrueNAS Backup Appliance, iXsystems servers, Project TrueView, and more – our booth was more popular than ever!

###End of life for NetBSD 6.x

In keeping with NetBSD’s policy of supporting only the latest (8.x) and next most recent (7.x) major branches, the recent release of NetBSD 8.0 marks the end of life for NetBSD 6.x. As in the past, a month of overlapping support has been provided in order to ease the migration to newer releases.

  • As of now, the following branches are no longer maintained:

  • netbsd-6-1

  • netbsd-6-0

  • netbsd-6

  • This means:

  • There will be no more pullups to those branches (even for security issues)

  • There will be no security advisories made for any those branches

  • The existing 6.x releases on ftp.NetBSD.org will be moved into /pub/NetBSD-archive/

  • May NetBSD 8.0 serve you well! (And if it doesn’t, please submit a PR!)

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Fri, 07 Sep 2018 11:00:00 -0700
Episode 262: OpenBSD Surfacing | BSD Now 262
OpenBSD on Microsoft Surface Go, FreeBSD Foundation August Update, What’s taking so long with Project Trident, pkgsrc config file versioning, and MacOS remnants in ZFS code.

##Headlines
OpenBSD on the Microsoft Surface Go

For some reason I like small laptops and the constraints they place on me (as long as they’re still usable). I used a Dell Mini 9 for a long time back in the netbook days and was recently using an 11" MacBook Air as my primary development machine for many years. Recently Microsoft announced a smaller, cheaper version of its Surface tablets called Surface Go which piqued my interest.

  • Hardware

The Surface Go is available in two hardware configurations: one with 4Gb of RAM and a 64Gb eMMC, and another with 8Gb of RAM with a 128Gb NVMe SSD. (I went with the latter.) Both ship with an Intel Pentium Gold 4415Y processor which is not very fast, but it’s certainly usable.
The tablet measures 9.65" across, 6.9" tall, and 0.3" thick. Its 10" diagonal 3:2 touchscreen is covered with Gorilla Glass and has a resolution of 1800x1200. The bezel is quite large, especially for such a small screen, but it makes sense on a device that is meant to be held, to avoid accidental screen touches.
The keyboard and touchpad are located on a separate, removable slab called the Surface Go Signature Type Cover which is sold separately. I opted for the “cobalt blue” cover which has a soft, cloth-like alcantara material. The cover attaches magnetically along the bottom edge of the device and presents USB-attached keyboard and touchpad devices. When the cover is folded up against the screen, it sends an ACPI sleep signal and is held to the screen magnetically. During normal use, the cover can be positioned flat on a surface or slightly raised up about 3/4" near the screen for better ergonomics. When using the device as a tablet, the cover can be rotated behind the screen which causes it to automatically stop sending keyboard and touchpad events until it is rotated back around.
The keyboard has a decent amount of key travel and a good layout, with Home/End/Page Up/Page Down being accessible via Fn+Left/Right/Up/Down but also dedicated Home/End/Page Up/Page Down keys on the F9-F12 keys which I find quite useful since the keyboard layout is somewhat small. By default, the F1-F12 keys do not send F1-F12 key codes and Fn must be used, either held down temporarily or Fn pressed by itself to enable Fn-lock which annoyingly keeps the bright Fn LED illuminated. The keys are backlit with three levels of adjustment, handled by the keyboard itself with the F7 key.
The touchpad on the Type Cover is a Windows Precision Touchpad connected via USB HID. It has a decent click feel but when the cover is angled up instead of flat on a surface, it sounds a bit hollow and cheap.

  • Surface Go Pen

The touchscreen is powered by an Elantech chip connected via HID-over-i2c, which also supports pen input. A Surface Pen digitizer is available separately from Microsoft and comes in the same colors as the Type Covers. The pen works without any pairing necessary, though the top button on it works over Bluetooth so it requires pairing to use. Either way, the pen requires an AAAA battery inside it to operate. The Surface Pen can attach magnetically to the left side of the screen when not in use.
A kickstand can swing out behind the display to use the tablet in a laptop form factor, which can adjust to any angle up to about 170 degrees. The kickstand stays firmly in place wherever it is positioned, which also means it requires a bit of force to pull it out when initially placing the Surface Go on a desk.
Along the top of the display are a power button and physical volume rocker buttons. Along the right side are the 3.5mm headphone jack, USB-C port, power port, and microSD card slot located behind the kickstand.
Charging can be done via USB-C or the dedicated charge port, which accommodates a magnetically-attached, thin barrel similar to Apple’s first generation MagSafe adapter. The charging cable has a white LED that glows when connected, which is kind of annoying since it’s near the mid-line of the screen rather than down by the keyboard. Unlike Apple’s MagSafe, the indicator light does not indicate whether the battery is charged or not. The barrel charger plug can be placed up or down, but in either direction I find it puts an awkward strain on the power cable coming out of it due to the vertical position of the port.
Wireless connectivity is provided by a Qualcomm Atheros QCA6174 802.11ac chip which also provides Bluetooth connectivity.
Most of the sensors on the device such as the gyroscope and ambient light sensor are connected behind an Intel Sensor Hub PCI device, which provides some power savings as the host CPU doesn’t have to poll the sensors all the time.

  • Firmware

The Surface Go’s BIOS/firmware menu can be entered by holding down the Volume Up button, then pressing and releasing the Power button, and releasing Volume Up when the menu appears. Secure Boot as well as various hardware components can be disabled in this menu. Boot order can also be adjusted. A temporary boot menu can be brought up the same way but using Volume Down instead.

###FreeBSD Foundation Update, August 2018

  • MESSAGE FROM THE EXECUTIVE DIRECTOR

Dear FreeBSD Community Member,
It’s been a busy summer for the Foundation. From traveling around the globe spreading the word about FreeBSD to bringing on new team members to improve the Project’s Continuous Integration work, we’re very excited about what we’ve accomplished. Take a minute to check out the latest updates within our Foundation sponsored projects; read more about our advocacy efforts in Bangladesh and community building in Cambridge; don’t miss upcoming Travel Grant deadlines, and new Developer Summits; and be sure to find out how your support will ensure our progress continues into 2019.
We can’t do this without you! Happy reading!! Deb

  • August 2018 Development Projects Update
  • Fundraising Update: Supporting the Project
  • August 2018 Release Engineering Update
  • BSDCam 2018 Recap
  • October 2018 FreeBSD Developer Summit Call for Participation
  • SANOG32 and COSCUP 2018 Recap
  • MeetBSD 2018 Travel Grant Application Deadline: September 7

##News Roundup
Project Trident: What’s taking so long?

  • What is taking so long?

The short answer is that it’s complicated.
Project Trident is quite literally a test of the new TrueOS build system. As expected, there have been quite a few bugs, undocumented features, and other optional bits that we discovered we needed that were not initially present. All of these things have to be addressed and retested in a constant back and forth process.
While Ken and JT are both experienced developers, neither has done this kind of release engineering before. JT has done some release engineering back in his Linux days, but the TrueOS and FreeBSD build system is very different. Both Ken and JT are learning a completely new way of building a FreeBSD/TrueOS distribution. Please keep in mind that no one has used this new TrueOS build system before, so Ken and JT want to not only provide a good Trident release, but also provide a model or template for other potential TrueOS distributions too!

  • Where are we now?

Through perseverance, trial and error, and a lot of head-scratching we have reached the point of having successful builds. It took a while to get there, but now we are simply working out a few bugs with the new installer that Ken wrote as well as finding and fixing all the new Xorg configuration options which recently landed in FreeBSD. We also found that a number of services have been removed or replaced between TrueOS 18.03 and 18.06 so we are needing to adjust what we consider the “base” services for the desktop. All of these issues are being resolved and we are continually rebuilding and pulling in new patches from TrueOS as soon as they are committed.
In the meantime we have made an early BETA release of Trident available to the users in our Telegram Channel for those who want to help out in testing these early versions.

  • Do you foresee any other delays?

At the moment we are doing many iterations of testing and tweaking the install ISO and package configurations in order to ensure that all the critical functionality works out-of-box (networking, sound, video, basic apps, etc). While we do not foresee any other major delays, sometimes things happen that our outside of our control. For an example, one of the recent delays that hit recently was completely unexpected: we had a hard drive failure on our build server. Up until recently, The aptly named “Poseidon” build server was running a Micron m500dc drive, but that drive is now constantly reporting errors. Despite ordering a replacement Western Digital Blue SSD several weeks ago, we just received it this past week. The drive is now installed with the builder back to full functionality, but we did lose many precious days with the delay.
The build server for Project Trident is very similar to the one that JT donated to the TrueOS project. JT had another DL580 G7, so he donated one to the Trident Project for their build server. Poseidon also has 256GB RAM (64 x 4GB sticks) which is a smidge higher than what the TrueOS builder has.
Since we are talking about hardware, we probably should address another question we get often, “What Hardware are the devs testing on?” So let’s go ahead and answer that one now.

  • Developer Hardware

  • JT: His main test box is a custom-built Intel i7 7700K system running 32GB RAM, dual Intel Optane 900P drives, and an Nvidia 1070 GTX with four 4K Acer Monitors. He also uses a Lenovo x250 ThinkPad alongside a desk full of x230t and x220 ThinkPads. One of which he gave away at SouthEast LinuxFest this year, which you can read about here. However it’s not done there, being a complete hardware hoarder, JT also tests on several Intel NUCs and his second laptop a Fujitsu t904, not to mention a Plethora of HP DL580 servers, a DL980 server, and a stack of BL485c, BL460c, and BL490c Blades in his HP c7000 and c3000 Bladecenter chassis. (Maybe it’s time for an intervention for his hardware collecting habits)

  • Ken: For a laptop, he primarily uses a 3rd generation X1 Carbon, but also has an old Eee PC T101MT Netbook (dual core 1GHz, 2GB of memory) which he uses for verifying how well Trident works on low-end hardware. As far as workstations go, his office computer is an Intel i7 with an NVIDIA Geforce GTX 960 running three 4K monitors and he has a couple other custom-built workstations (1 AMD, 1 Intel+NVIDIA) at his home. Generally he assembled random workstations based on hardware that was given to him or that he could acquire cheap.

  • Tim: is using a third gen X1 Carbon and a custom built desktop with an Intel Core i5-4440 CPU, 16 GiB RAM, Nvidia GeForce GTX 750 Ti, and a RealTek 8168 / 8111 network card.

  • Rod: Rod uses… No one knows what Rod uses, It’s kinda like how many licks does it take to get to the center of a Tootsie-Roll Tootsie-Pop… the world may just never know.

###NetBSD GSoC: pkgsrc config file versioning

Packages may install code (both machine executable code and interpreted programs), documentation and manual pages, source headers, shared libraries and other resources such as graphic elements, sounds, fonts, document templates, translations and configuration files, or a combination of them.
Configuration files are usually the means through which the behaviour of software without a user interface is specified. This covers parts of the operating systems, network daemons and programs in general that don’t come with an interactive graphical or textual interface as the principal mean for setting options.
System wide configuration for operating system software tends to be kept under /etc, while configuration for software installed via pkgsrc ends up under LOCALBASE/etc (e.g., /usr/pkg/etc).
Software packaged as part of pkgsrc provides example configuration files, if any, which usually get extracted to LOCALBASE/share/examples/PKGBASE/.
Don’t worry: automatic merging is disabled by default, set $VCSAUTOMERGE to enable it.
In order to avoid breakage, installed configuration is backed up first in the VCS, separating user-modified files from files that have been already automatically merged in the past, in order to allow the administrator to easily restore the last manually edited file in case of breakage.
VCS functionality only applies to configuration files, not to rc.d scripts, and only if the environment variable $NOVCS is unset.
The version control system to be used as a backend can be set through $VCS. It default to RCS, the Revision Control System, which works only locally and doesn’t support atomic transactions.
Other backends such as CVS are supported and more will come; these, being used at the explicit request of the administrator, need to be already installed and placed in a directory part of $PATH.

pkgsrc is now able to deploy configuration from packages being installed from a remote, site-specific vcs repository.
User modified files are always tracked even if automerge functionality is not enabled, and a new tool, pkgconftrack(1), exists to manually store user changes made outside of package upgrade time.
Version Control software is executed as the same user running pkgadd or make install, unless the user is “root”. In this case, a separate, unprivileged user, pkgvcsconf, gets created with its own home directory and a working login shell (but no password). The home directory is not strictly necessary, it exists to facilitate migrations betweens repositories and vcs changes; it also serves to store keys used to access remote repositories.
Using git instead of rcs is simply done by setting VCS=git in pkginstall.conf

Support for configuration tracking is in scripts, pkginstall scripts, that get built into binary packages and are run by pkgadd upon installation. The idea behind the proposal suggested that users of the new feature should be able to store revisions of their installed configuration files, and of package-provided default, both in local or remote repositories. With this capability in place, it doesn’t take much to make the scripts “pull” configuration from a VCS repository at installation time.
That’s what setting VCSCONFPULL=yes in pkginstall.conf after having enabled VCSTRACKCONF does: You are free to use official, third party prebuilt packages that have no customization in them, enable these options, and point pkgsrc to a private conf repository. If it contains custom configuration for the software you are installing, an attempt will be made to use it and install it on your system. If it fails, pkginstall will fall back to using the defaults that come inside the package. RC scripts are always deployed from the binary package, if existing and PKGRCDSCRIPTS=yes in pkginstall.conf or the environment.
This will be part of packages, not a separate solution like configuration management tools. It doesn’t support running scripts on the target system to customize the installation, it doesn’t come with its domain-specific language, it won’t run as a daemon or require remote logins to work. It’s quite limited in scope, but you can define a ROLE for your system in pkginstall.conf or in the environment, and pkgsrc will look for configuration you or your organization crafted for such a role (e.g., public, standalone webserver vs reverse proxy or node in a database cluster)

###A little bit of the one-time MacOS version still lingers in ZFS

Once upon a time, Apple came very close to releasing ZFS as part of MacOS. Apple did this work in its own copy of the ZFS source base (as far as I know), but the people in Sun knew about it and it turns out that even today there is one little lingering sign of this hoped-for and perhaps prepared-for ZFS port in the ZFS source code. Well, sort of, because it’s not quite in code.
Lurking in the function that reads ZFS directories to turn (ZFS) directory entries into the filesystem independent format that the kernel wants is the following comment:

objnum = ZFSDIRENTOBJ(zap.zafirstinteger);
/
MacOS X can extract the object type here such as:
* uint8t type = ZFSDIRENTTYPE(zap.zafirstinteger);
*/

  • Specifically, this is in zfsreaddir in zfsvnops.c .

ZFS maintains file type information in directories. This information can’t be used on Solaris (and thus Illumos), where the overall kernel doesn’t have this in its filesystem independent directory entry format, but it could have been on MacOS (‘Darwin’), because MacOS is among the Unixes that support d_type. The comment itself dates all the way back to this 2007 commit, which includes the change ‘reserve bits in directory entry for file type’, which created the whole setup for this.
I don’t know if this file type support was added specifically to help out Apple’s MacOS X port of ZFS, but it’s certainly possible, and in 2007 it seems likely that this port was at least on the minds of ZFS developers. It’s interesting but understandable that FreeBSD didn’t seem to have influenced them in the same way, at least as far as comments in the source code go; this file type support is equally useful for FreeBSD, and the FreeBSD ZFS port dates to 2007 too (per this announcement).
Regardless of the exact reason that ZFS picked up maintaining file type information in directory entries, it’s quite useful for people on both FreeBSD and Linux that it does so. File type information is useful for any number of things and ZFS filesystems can (and do) provide this information on those Unixes, which helps make ZFS feel like a truly first class filesystem, one that supports all of the expected general system features.

##Beastie Bits

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 06 Sep 2018 02:00:00 -0700
Episode 261: FreeBSDcon Flashback | BSD Now 261
Insight into TrueOS and Trident, stop evildoers with pf-badhost, Flashback to FreeBSDcon ‘99, OpenBSD’s measures against TLBleed, play Morrowind on OpenBSD in 5 steps, DragonflyBSD developers shocked at Threadripper performance, and more.

##Headlines
An Insight into the Future of TrueOS BSD and Project Trident

Last month, TrueOS announced that they would be spinning off their desktop offering. The team behind the new project, named Project Trident, have been working furiously towards their first release. They did take a few minutes to answer some of our question about Project Trident and TrueOS. I would like to thank JT and Ken for taking the time to compile these answers.

  • It’s FOSS: What is Project Trident?

Project Trident: Project Trident is the continuation of the TrueOS Desktop. Essentially, it is the continuation of the primary “TrueOS software” that people have been using for the past 2 years. The continuing evolution of the entire TrueOS project has reached a stage where it became necessary to reorganize the project. To understand this change, it is important to know the history of the TrueOS project.

Originally, Kris Moore created PC-BSD. This was a Desktop release of FreeBSD focused on providing a simple and user-friendly graphical experience for FreeBSD. PC-BSD grew and matured over many years. During the evolution of PC-BSD, many users began asking for a server focused version of the software. Kris agreed, and TrueOS was born as a scaled down server version of PC-BSD. In late 2016, more contributors and growth resulted in significant changes to the PC-BSD codebase. Because the new development was so markedly different from the original PC-BSD design, it was decided to rebrand the project.

TrueOS was chosen as the name for this new direction for PC-BSD as the project had grown beyond providing only a graphical front to FreeBSD and was beginning to make fundamental changes to the FreeBSD operating system. One of these changes was moving PC-BSD from being based on each FreeBSD Release to TrueOS being based on the active and less outdated FreeBSD Current. Other major changes are using OpenRC for service management and being more aggressive about addressing long-standing issues with the FreeBSD release process. TrueOS moved toward a rolling release cycle, twice a year, which tested and merged FreeBSD changes directly from the developer instead of waiting months or even years for the FreeBSD review process to finish. TrueOS also deprecated and removed obsolete technology much more regularly.

As the TrueOS Project grew, the developers found these changes were needed by other FreeBSD-based projects. These projects began expressing interest in using TrueOS rather than FreeBSD as the base for their project. This demonstrated that TrueOS needed to again evolve into a distribution framework for any BSD project to use. This allows port maintainers and source developers from any BSD project to pool their resources and use the same source repositories while allowing every distribution to still customize, build, and release their own self-contained project. The result is a natural split of the traditional TrueOS team. There were now naturally two teams in the TrueOS project: those working on the build infrastructure and FreeBSD enhancements – the “core” part of the project, and those working on end-user experience and utility – the “desktop” part of the project.

When the decision was made to formally split the projects, the obvious question that arose was what to call the “Desktop” project. As TrueOS was already positioned to be a BSD distribution platform, the developers agreed the desktop side should pick a new name. There were other considerations too, one notable being that we were concerned that if we continued to call the desktop project “TrueOS Desktop”, it would prevent people from considering TrueOS as the basis for their distribution because of misconceptions that TrueOS was a desktop-focused OS. It also helps to “level the playing field” for other desktop distributions like GhostBSD so that TrueOS is not viewed as having a single “blessed” desktop version.

  • It’s FOSS: What features will TrueOS add to the FreeBSD base?

Project Trident: TrueOS has already added a number of features to FreeBSD:
OpenRC replaces rc.d for service management
LibreSSL in base
Root NSS certificates out-of-box
Scriptable installations (pc-sysinstall)
The full list of changes can be seen on the TrueOS repository (https://github.com/trueos/trueos/blob/trueos-master/README.md). This list does change quite regularly as FreeBSD development itself changes.

  • It’s FOSS: I understand that TrueOS will have a new feature that will make creating a desktop spin of TrueOS very easy. Could you explain that new feature?

Project Trident: Historically, one of the biggest hurdles for creating a desktop version of FreeBSD is that the build options for packages are tuned for servers rather than desktops. This means a desktop distribution cannot use the pre-built packages from FreeBSD and must build, use, and maintain a custom package repository. Maintaining a fork of the FreeBSD ports tree is no trivial task. TrueOS has created a full distribution framework so now all it takes to create a custom build of FreeBSD is a single JSON manifest file. There is now a single “source of truth” for the source and ports repositories that is maintained by the TrueOS team and regularly tagged with “stable” build markers. All projects can use this framework, which makes updates trivial.

  • It’s FOSS: Do you think that the new focus of TrueOS will lead to the creation of more desktop-centered BSDs?

Project Trident: That is the hope. Historically, creating a desktop-centered BSD has required a lot of specialized knowledge. Not only do most people not have this knowledge, but many do not even know what they need to learn until they start troubleshooting. TrueOS is trying to drastically simplify this process to enable the wider Open Source community to experiment, contribute, and enjoy BSD-based projects.

  • It’s FOSS: What is going to happen to TrueOS Pico? Will Project Trident have ARM support?

Project Trident: Project Trident will be dependent on TrueOS for ARM support. The developers have talked about the possibility of supporting ARM64 and RISC-V architectures, but it is not possible at the current time. If more Open Source contributors want to help develop ARM and RISC-V support, the TrueOS project is definitely willing to help test and integrate that code.

  • It’s FOSS: What does this change (splitting Trus OS into Project Trident) mean for the Lumina desktop environment?

Project Trident: Long-term, almost nothing. Lumina is still the desktop environment for Project Trident and will continue to be developed and enhanced alongside Project Trident just as it was for TrueOS. Short-term, we will be delaying the release of Lumina 2.0 and will release an updated version of the 1.x branch (1.5.0) instead. This is simply due to all the extra overhead to get Project Trident up and running. When things settle down into a rhythm, the development of Lumina will pick up once again.

  • It’s FOSS: Are you planning on including any desktop environments besides Lumina?

Project Trident: While Lumina is included by default, all of the other popular desktop environments will be available in the package repo exactly as they had been before.

  • It’s FOSS: Any plans to include Steam to increase the userbase?

Project Trident: Steam is still unavailable natively on FreeBSD, so we do not have any plans to ship it out of the box currently. In the meantime, we highly recommend installing the Windows version of Steam through the PlayOnBSD utility.

  • It’s FOSS: What will happen to the AppCafe?

Project Trident: The AppCafe is the name of the graphical interface for the “pkg” utility integrated into the SysAdm client created by TrueOS. This hasn’t changed. SysAdm, the graphical client, and by extension AppCafe are still available for all TrueOS-based distributions to use.

  • It’s FOSS: Does Project Trident have any corporate sponsors lined up? If not, would you be open to it or would you prefer that it be community supported?

Project Trident: iXsystems is the first corporate sponsor of Project Trident and we are always open to other sponsorships as well. We would prefer smaller individual contributions from the community, but we understand that larger project needs or special-purpose goals are much more difficult to achieve without allowing larger corporate sponsorships as well. In either case, Project Trident is always looking out for the best interests of the community and will not allow intrusive or harmful code to enter the project even if a company or individual tries to make that code part of a sponsorship deal.

  • It’s FOSS: BSD always seems to be lagging in terms of support for newer devices. Will TrueOS be able to remedy that with a quicker release cycle?

Project Trident: Yes! That was a primary reason for TrueOS to start tracking the CURRENT branch of FreeBSD in 2016. This allows for the changes that FreeBSD developers are making, including new hardware support, to be available much sooner than if we followed the FreeBSD release cycle.

  • It’s FOSS: Do you have any idea when Project Trident will have its first release?

Project Trident: Right now we are targeting a late August release date. This is because Project Trident is “kicking the wheels” on the new TrueOS distribution system. We want to ensure everything is working smoothly before we release. Going forward, we plan on having regular package updates every week or two for the end-user packages and a new release of Trident with an updated OS version every 6 months. This will follow the TrueOS release schedule with a small time offset.

###pf-badhost: Stop the evil doers in their tracks!

pf-badhost is a simple, easy to use badhost blocker that uses the power of the pf firewall to block many of the internet’s biggest irritants. Annoyances such as ssh bruteforcers are largely eliminated. Shodan scans and bots looking for webservers to abuse are stopped dead in their tracks. When used to filter outbound traffic, pf-badhost blocks many seedy, spooky malware containing and/or compromised webhosts.
Filtering performance is exceptional, as the badhost list is stored in a pf table. To quote the OpenBSD FAQ page regarding tables: “the lookup time on a table holding 50,000 addresses is only slightly more than for one holding 50 addresses.”
pf-badhost is simple and powerful. The blocklists are pulled from quality, trusted sources. The ‘Firehol’, ‘Emerging Threats’ and ‘Binary Defense’ block lists are used as they are popular, regularly updated lists of the internet’s most egregious offenders. The pf-badhost.sh script can easily be expanded to use additional or alternate blocklists.
pf-badhost works best when used in conjunction with unbound-adblock for the ultimate badhost blocking.

  • Notes:
  • If you are trying to run pf-badhost on a LAN or are using NAT, you will want to add a rule to your pf.conf appearing BEFORE the pf-badhost rules allowing traffic to and from your local subnet so that you can still access your gateway and any DNS servers.
  • Conversely, adding a line to pf-badhost.sh that removes your subnet range from the <pfbadhost> table should also work. Just make sure you choose a subnet range / CIDR block that is actually in the list. 192.168.0.0/16, 172.16.0.0/12 and 10.0.0.0/8 are the most common home/office subnet ranges.

DigitalOcean
https://do.co/bsdnow

###FLASHBACK: FreeBSDCon’99: Fans of Linux’s lesser-known sibling gather for the first time

FreeBSD, a port of BSD Unix to Intel, has been around almost as long as Linux has – but without the media hype. Its developer and user community recently got a chance to get together for the first time, and they did it in the city where BSD – the Berkeley Software Distribution – was born some 25 years ago.
October 17, 1999 marked a milestone in the history of FreeBSD – the first FreeBSD conference was held in the city where it all began, Berkeley, CA. Over 300 developers, users, and interested parties attended from around the globe.
This was easily 50 percent more people than the conference organizers had expected. This first conference was meant to be a gathering mostly for developers and FreeBSD advocates. The turnout was surprisingly (and gratifyingly) large.
In fact, attendance exceeded expectations so much that, for instance, Kirk McKusick had to add a second, identical tutorial on FreeBSD internals, because it was impossible for everyone to attend the first!
But for a first-ever conference, I was impressed by how smoothly everything seemed to go. Sessions started on time, and the sessions I attended were well-run; nothing seemed to be too cold, dark, loud, late, or off-center.
Of course, the best part about a conference such as this one is the opportunity to meet with other people who share similar interests. Lunches and breaks were a good time to meet people, as was the Tuesday night beer bash.
The Wednesday night reception was of a type unusual for the technical conferences I usually attend – a three-hour Hornblower dinner cruise on San Francisco Bay. Not only did we all enjoy excellent food and company, but we all got to go up on deck and watch the lights of San Francisco and Berkeley as we drifted by. Although it’s nice when a conference attracts thousands of attendees, there are some things that can only be done with smaller groups of people; this was one of them.
In short, this was a tiny conference, but a well-run one.

  • Sessions

Although it was a relatively small conference, the number and quality of the sessions belied the size. Each of the three days of the conference featured a different keynote speaker. In addition to Jordan Hubbard, Jeremy Allison spoke on “Samba Futures” on day two, and Brian Behlendorf gave a talk on “FreeBSD and Apache: A Perfect Combo” to start off the third day.
The conference sessions themselves were divided into six tracks: advocacy, business, development, networking, security, and panels. The panels track featured three different panels, made up of three different slices of the community: the FreeBSD core team, a press panel, and a prominent user panel with representatives from such prominent commercial users as Yahoo! and USWest.
I was especially interested in Apple Computer’s talk in the development track. Wilfredo Sanchez, technical lead for open source projects at Apple (no, that’s not an oxymoron!) spoke about Apple’s Darwin project, the company’s operating system road map, and the role of BSD (and, specifically, FreeBSD) in Apple’s plans.
Apple and Unix have had a long and uneasy history, from the Lisa through the A/UX project to today. Personally, I’m very optimistic about the chances for the Darwin project to succeed. Apple’s core OS kernel team has chosen FreeBSD as its reference platform. I’m looking forward to what this partnership will bring to both sides.
Other development track sessions included in-depth tutorials on writing device drivers, basics of the Vinum Volume Manager, Fibre Channel, development models (the open repository model), and the FreeBSD Documentation Project (FDP). If you’re interested in contributing to the FreeBSD project, the FDP is a good place to start.
Advocacy sessions included “How One Person Can Make a Difference” (a timeless topic that would find a home at any technical conference!) and “Starting and Managing A User Group” (trials and tribulations as well as rewards).
The business track featured speakers from three commercial users of FreeBSD: Cybernet, USWest, and Applix. Applix presented its port of Applixware Office for FreeBSD and explained how Applix has taken the core services of Applixware into open source.
Commercial applications and open source were once a rare combination; we can only hope the trend away from that state of affairs will continue.

  • Commercial use of FreeBSD

The use of FreeBSD in embedded applications is increasing as well – and it is increasing at the same rate that hardware power is. These days, even inexpensive systems are able to run a BSD kernel.
The BSD license and the solid TCP/IP stack prove significant enticements to this market as well. (Unlike the GNU Public License, the BSD license does not require that vendors make derivative works open source.)
Companies such as USWest and Verio use FreeBSD for a wide variety of different Internet services.
Yahoo! and Hotmail are examples of companies that use FreeBSD extensively for more specific purposes. Yahoo!, for example, has many hundreds of FreeBSD boxes, and Hotmail has almost 2000 FreeBSD machines at its data center in the San Francisco Bay area.
Hotmail is owned by Microsoft, so the fact that it runs FreeBSD is a secret. Don’t tell anyone…
When asked to comment on the increasing commercial interest in BSD, Hubbard said that FreeBSD is learning the Red Hat lesson. “Walnut Creek and others with business interests in FreeBSD have learned a few things from the Red Hat IPO,” he said, “and nobody is just sitting around now, content with business as usual. It’s clearly business as unusual in the open source world today.”
Hubbard had also singled out some of BSD’s commercial partners, such as Whistle Communications, for praise in his opening day keynote. These partners play a key role in moving the project forward, he said, by contributing various enhancements and major new systems, such as Netgraph, as well as by contributing paid employee time spent on FreeBSD.
Even short FreeBSD-related contacts can yield good results, Hubbard said. An example of this is the new jail() security code introduced in FreeBSD 3.x and 4.0, which was contributed by R & D Associates. A number of ISPs are also now donating the hardware and bandwidth that allows the project to provide more resource mirrors and experimental development sites.

  • See you next year

And speaking of corporate sponsors, thanks go to Walnut Creek for sponsoring the conference, and to Yahoo! for covering all the expenses involved in bringing the entire FreeBSD core team to Berkeley.
As a fan of FreeBSD, I’m happy to see that the project has finally produced a conference. It was time: many of the 16 core team members had been working together on a regular basis for nearly seven years without actually meeting face to face.
It’s been an interesting year for open source projects. I’m looking forward to the next year – and the next BSD conference – to be even better.

##News Roundup
OpenBSD Recommends: Disable SMT/Hyperthreading in all Intel BIOSes

Two recently disclosed hardware bugs affected Intel cpus: - TLBleed - T1TF (the name "Foreshadow" refers to 1 of 3 aspects of this bug, more aspects are surely on the way) Solving these bugs requires new cpu microcode, a coding workaround, *AND* the disabling of SMT / Hyperthreading. SMT is fundamentally broken because it shares resources between the two cpu instances and those shared resources lack security differentiators. Some of these side channel attacks aren't trivial, but we can expect most of them to eventually work and leak kernel or cross-VM memory in common usage circumstances, even such as javascript directly in a browser. There will be more hardware bugs and artifacts disclosed. Due to the way SMT interacts with speculative execution on Intel cpus, I expect SMT to exacerbate most of the future problems. A few months back, I urged people to disable hyperthreading on all Intel cpus. I need to repeat that: DISABLE HYPERTHREADING ON ALL YOUR INTEL MACHINES IN THE BIOS. Also, update your BIOS firmware, if you can. OpenBSD -current (and therefore 6.4) will not use hyperthreading if it is enabled, and will update the cpu microcode if possible. But what about 6.2 and 6.3? The situation is very complex, continually evolving, and is taking too much manpower away from other tasks. Furthermore, Intel isn't telling us what is coming next, and are doing a terrible job by not publically documenting what operating systems must do to resolve the problems. We are having to do research by reading other operating systems. There is no time left to backport the changes -- we will not be issuing a complete set of errata and syspatches against 6.2 and 6.3 because it is turning into a distraction. Rather than working on every required patch for 6.2/6.3, we will re-focus manpower and make sure 6.4 contains the best solutions possible. So please try take responsibility for your own machines: Disable SMT in the BIOS menu, and upgrade your BIOS if you can. I'm going to spend my money at a more trustworthy vendor in the future.

###Get Morrowind running on OpenBSD in 5 simple steps

This article contains brief instructions on how to get one of the greatest Western RPGs of all time, The Elder Scrolls III: Morrowind, running on OpenBSD using the OpenMW open source engine recreation. These instructions were tested on a ThinkPad X1 Carbon Gen 3. The information was adapted from this OpenMW forum thread: https://forum.openmw.org/viewtopic.php?t=3510

  • Purchase and download the DRM-free version from GOG (also considered the best version due to the high quality PDF guide that it comes with): https://www.gog.com/game/theelderscrollsiiimorrowindgotyedition
  • Install the required packages built from the ports tree as root. openmw is the recreated game engine, and innoextract is how we will get the game data files out of the win32 executable.

pkgadd openmw innoextract

  • Move the file from GOG setuptesmorrowindgoty2.0.0.7.exe into its own directory morrowind/ due to innoextract’s default behaviour of extracting into the current directory. Then type:

innoextract setuptesmorrowindgoty2.0.0.7.exe

  • Type openmw-wizard and follow the straightforward instructions. Note that you have a pre-existing installation, and select the morrowind/app/Data Files folder that innoextract extracted.
  • Type in openmw-launcher, toggle the settings to your preferences, and then hit play!

iXsystems
https://twitter.com/allanjude/status/1034647571124367360

###My First Clang Bug

Part of the role of being a packager is compiling lots (and lots) of packages. That means compiling lots of code from interesting places and in a variety of styles. In my opinion, being a good packager also means providing feedback to upstream when things are bad. That means filing upstream bugs when possible, and upstreaming patches.
One of the “exciting” moments in packaging is when tools change. So each and every major CMake update is an exercise in recompiling 2400 or more packages and adjusting bits and pieces. When a software project was last released in 2013, adjusting it to modern tools can become quite a chore (e.g. Squid Report Generator). CMake is excellent for maintaining backwards compatibility, generally accommodating old software with new policies. The most recent 3.12 release candidate had three issues filed from the FreeBSD side, all from fallout with older software. I consider the hours put into good bug reports, part of being a good citizen of the Free Software world.
My most interesting bug this week, though, came from one line of code somewhere in Kleopatra: QUNUSED(gpgagentdata);
That one line triggered a really peculiar link error in KDE’s FreeBSD CI system. Yup … telling the compiler something is unused made it fall over. Commenting out that line got rid of the link error, but introduced a warning about an unused function. Working with KDE-PIM’s Volker Krause, we whittled the problem down to a six-line example program — two lines if you don’t care much for coding style. I’m glad, at that point, that I could throw it over the hedge to the LLVM team with some explanatory text. Watching the process on their side reminds me ever-so-strongly of how things work in KDE (or FreeBSD for that matter): Bugzilla, Phabricator, and git combine to be an effective workflow for developers (perhaps less so for end-users).
Today I got a note saying that the issue had been resolved. So brief a time for a bug. Live fast. Get squashed young.

###DragonFlyBSD Now Runs On The Threadripper 2990WX, Developer Shocked At Performance

Last week I carried out some tests of BSD vs. Linux on the new 32-core / 64-thread Threadripper 2990WX. I tested FreeBSD 11, FreeBSD 12, and TrueOS – those benchmarks will be published in the next few days. I tried DragonFlyBSD, but at the time it wouldn’t boot with this AMD HEDT processor. But now the latest DragonFlyBSD development kernel can handle the 2990WX and the lead DragonFly developer calls this new processor “a real beast” and is stunned by its performance potential.

When I tried last week, the DragonFlyBSD 5.2.2 stable release nor DragonFlyBSD 5.3 daily snapshot would boot on the 2990WX. But it turns out Matthew Dillon, the lead developer of DragonFlyBSD, picked up a rig and has it running now. So in time for the next 5.4 stable release or those using the daily snapshots can have this 32-core / 64-thread Zen+ CPU running on this operating system long ago forked from FreeBSD.

In announcing his success in bringing up the 2990WX under DragonFlyBSD, which required a few minor changes, he shared his performance thoughts and hopes for the rig. “The cpu is a real beast, packing 32 cores and 64 threads. It blows away our dual-core Xeon to the tune of being +50% faster in concurrent compile tests, and it also blows away our older 4-socket Opteron (which we call ‘Monster’) by about the same margin. It’s an impressive CPU. For now the new beast is going to be used to help us improve I/O performance through the filesystem, further SMP work (but DFly scales pretty well to 64 threads already), and perhaps some driver to work to support the 10gbe on the mobo.”

Dillon shared some results on the system as well. " The Threadripper 2990WX is a beast. It is at least 50% faster than both the quad socket opteron and the dual socket Xeon system I tested against. The primary limitation for the 2990WX is likely its 4 channels of DDR4 memory, and like all Zen and Zen+ CPUs, memory performance matters more than CPU frequency (and costs almost no power to pump up the performance). That said, it still blow away a dual-socket Xeon with 3x the number of memory channels. That is impressive!"

The well known BSD developer also added, “This puts the 2990WX at par efficiency vs a dual-socket Xeon system, and better than the dual-socket Xeon with slower memory and a power cap. This is VERY impressive. I should note that the 2990WX is more specialized with its asymetric NUMA architecture and 32 cores. I think the sweet spot in terms of CPU pricing and efficiency is likely going to be with the 2950X (16-cores/32-threads). It is clear that the 2990WX (32-cores/64-threads) will max out 4-channel memory bandwidth for many workloads, making it a more specialized part. But still awesome…This thing is an incredible beast, I’m glad I got it.”

While I have the FreeBSD vs. Linux benchmarks from a few days ago, it looks like now on my ever growing TODO list will be re-trying out the newest DragonFlyBSD daily snapshot for seeing how the performance compares in the mix. Stay tuned for the numbers that should be in the next day or two.

##Beastie Bits

Tarsnap

##Feedback/Questions

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 30 Aug 2018 00:00:00 -0700
Episode 260: Hacking Tour of Europe | BSD Now 260
Trip reports from the Essen Hackathon and BSDCam, CfT: ZFS native encryption and UFS trim consolidation, ZFS performance benchmarks on a FreeBSD server, how to port your OS to EC2, Vint Cerf about traceability, Remote Access console to an RPi3 running FreeBSD, and more.

##Headlines
Essen Hackathon & BSDCam 2018 trip report

  • Allan and Benedict met at FRA airport and then headed to the Air Rail terminal for our train to Essen where the Hackathon would happen over the weekend of Aug 10 - 12, 2018. Once there, we did not have to wait long until other early-arrivals would show up and soon we had about 10 people gathered for lunch. After buying some take-out pizzas and bringing it back to the Linuxhotel (there was a training still going on there so we could not get into our rooms yet), we sat in the sunny park and talked. More and more people arrived and soon, people started hacking on their laptops. Some people would not arrive until a few hours before midnight, but we already had a record appearance of 20 people in total.
  • On Saturday, we gathered everyone in one of the seminar rooms that had rooms and chairs for us. After some organizational infos, we did an introductory round and Benedict wrote down on the whiteboard what people were interested in. It was not long until groups formed to talk about SSL in base, weird ZFS scrubs that would go over 100% completion (fixed now). Other people started working on ports, fixing bugs, or wrote documentation. The day ended in a BBQ in the Linuxhotel park, which was well received by everyone.
  • On Sunday, after attendees packed up their luggage and stored it in the seminar room, we continued hacking until lunchtime. After a quick group picture, we headed to a local restaurant for the social event (which was not open on Saturday, otherwise we would have had it then). In the afternoon, most people departed, a good half of them were heading for BSDCam.
  • Commits from the hackathon (the ones from 2018)
  • Overall, the hackathon was well received by attendees and a lot of them liked the fact that it was close to another BSD gathering so they could nicely combine the two. Also, people thought about doing their own hackathon in the future, which is an exciting prospect. Thanks to all who attended, helped out here and there when needed. Special Thanks to Netzkommune GmbH for sponsoring the social event and the Linuxhotel for having us.
  • Benedict was having a regular work day on Monday after coming back from the hackathon, but flew out to Heathrow on Tuesday. Allan was in London a day earlier and arrived a couple of hours before Benedict in Cambridge. He headed for the Computer Lab even though the main event would not start until Wednesday. Most people gathered at the Maypole pub on Tuesday evening for welcomes, food and drinks.
  • On Wednesday, a lot of people met in the breakfast room of Churchill College where most people were staying and went to the Computer Lab, which served as the main venue for BSDCam, together. The morning was spend with introductions and collecting what most people were interested in talking. This unconference style has worked well in the past and soon we had 10 main sessions together for the rest of this and the following two days (full schedule).
  • Most sessions took notes, which you can find on the FreeBSD wiki.
  • On Thursday evening, we had a nice formal dinner at Trinity Hall.
  • BSDCam 2018 was a great success with a lot of fruitful discussions and planning sessions. We thank the organizers for BSDCam for making it happen.
  • A special mentions goes out to Robert Watson and his family. Even though he was not there, he had a good reason to miss it: they had their first child born at the beginning of the week. Congratulations and best wishes to all three of them!

###Call for Testing: ZFS Native Encryption for FreeBSD

iXsystems

###Call for Testing: UFS TRIM Consolidation

  • Kirk Mckusick posts to the FreeBSD mailing list looking for testers for the new UFS TRIM Consolidation code

When deleting files on filesystems that are stored on flash-memory (solid-state) disk drives, the filesystem notifies the underlying disk of the blocks that it is no longer using. The notification allows the drive to avoid saving these blocks when it needs to flash (zero out) one of its flash pages. These notifications of no-longer-being-used blocks are referred to as TRIM notifications. In FreeBSD these TRIM notifications are sent from the filesystem to the drive using the BIODELETE command.
Until now, the filesystem would send a separate message to the drive for each block of the file that was deleted. Each Gigabyte of file size resulted in over 3000 TRIM messages being sent to the drive. This burst of messages can overwhelm the drive’s task queue causing multiple second delays for read and write requests.
This implementation collects runs of contiguous blocks in the file and then consolodates them into a single BIODELETE command to the drive. The BIODELETE command describes the run of blocks as a single large block being deleted. Each Gigabyte of file size can result in as few as two BIODELETE commands and is typically less than ten. Though these larger BIODELETE commands take longer to run, they do not clog the drive task queue, so read and write commands can intersperse effectively with them.
Though this new feature has been throughly reviewed and tested, it is being added disabled by default so as to minimize the possibility of disrupting the upcoming 12.0 release. It can be enabled by running `sysctl vfs.ffs.dotrimcons=1’’. Users are encouraged to test it. If no problems arise, we will consider requesting that it be enabled by default for 12.0.
This support is off by default, but I am hoping that I can get enough testing to ensure that it (a) works, and (b) is helpful that it will be reasonable to have it turned on by default in 12.0. The cutoff for turning it on by default in 12.0 is September 19th. So I am requesting your testing feedback in the near-term. Please let me know if you have managed to use it successfully (or not) and also if it provided any performance difference (good or bad).

##News Roundup
ZFS performance

  • Aravindh Sampathkumar, a Performance Engineer and Sysadmin posts some simple benchmarks he did on a new ZFS server

This is NOT an all-in post about ZFS performance. I built a FreeBSD+ZFS file server recently at work to serve as an offsite backup server. I wanted to run a few synthetic workloads on it and look at how it fares from performance perspective. Mostly for curiosity and learning purposes.
As stated in the notes about building this server, performance was not one of the priorities, as this server will never face our active workload. What I care about from this server is its ability to work with rsync and keep the data synchronised with our primary storage server. With that context, I ran a few write tests to see how good our solution is and what to expect from it in terms of performance.

  • The article then uses FIO to do some benchmarks.
  • As the author did, make sure you match the FIO block size to the ZFS record size to avoid write amplification. Either tune FIO or adjust the recordsize property in ZFS
  • You also want to consider compression and cache effects

Write Performance: Incompressible: 1600-2600 MB/s, Compressible: 2500-6600 MB/s
Another over 1200 MB/s is enough to keep your 10 gigabit network saturated

  • The increased latency that is seen with higher number of writers working, may be the result of the ZFS backpressure system (the write throttle). There is some tuning that can be done there. Specifically, since this machine has 768 GB of ram, you might allow more than 4GB of dirty data, which would mean you’d be able to write larger batches and not have to push back while you wait for a transaction group to flush when dealing with gigabytes/sec of writes

###How to port your OS to EC2

  • Colin Percival reflects on his FreeBSD on EC2 maintainership efforts in his blog:

I’ve been the maintainer of the FreeBSD/EC2 platform for about 7.5 years now, and as far as “running things in virtual machines” goes, that remains the only operating system and the only cloud which I work on. That said, from time to time I get questions from people who want to port other operating systems into EC2, and being a member of the open source community, I do my best to help them. I realized a few days ago that rather than replying to emails one by one it would be more efficient to post something publicly; so — for the benefit of the dozen or so people who want to port operating systems to run in EC2, and the curiosity of maybe a thousand more people who use EC2 but will never build AMIs themselves — here’s a rough guide to building EC2 images.
Before we can talk about building images, there are some things you need:
Your OS needs to run on x86 hardware. 64-bit (“amd64”, “x86-64”) is ideal, but I’ve managed to run 32-bit FreeBSD on “64-bit” EC2 instances so at least in some cases that’s not strictly necessary.
You almost certainly want to have drivers for Xen block devices (for all of the pre-Nitro EC2 instances) or for NVMe disks (for the most recent EC2 instances). Theoretically you could make do without these since there’s some ATA emulation available for bootstrapping, but if you want to do any disk I/O after the kernel finishes booting you’ll want to have a disk driver.
Similarly, you need support for the Xen network interface (older instances), Intel 10 GbE SR-IOV networking (some newer but pre-Nitro instances), or Amazon’s “ENA” network adapters (on Nitro instances), unless you plan on having instances which don’t communicate over the network. The ENA driver is probably the hardest thing to port, since as far as I know there’s no way to get your hands on the hardware directly, and it’s very difficult to do any debugging in EC2 without having a working network.
Finally, the obvious: You need to have an AWS account, and appropriate API access keys.
Building a disk image

Building an AMI
I wrote a simple tool for converting disk images into EC2 instances: bsdec2-image-upload. It uploads a disk image to Amazon S3; makes an API call to import that disk image into an EBS volume; creates a snapshot of that volume; then registers an EC2 AMI using that snapshot.
To use bsdec2-image-upload, you’ll first need to create an S3 bucket for it to use as a staging area. You can call it anything you like, but I recommend that you

Create it in a “nearby” region (for performance reasons), and
Set an S3 “lifecycle policy” which deletes objects automatically after 1 day (since bsdec2-image-upload doesn’t clean up the S3 bucket, and those objects are useless once you’ve finished creating an AMI).

Boot configuration
Odds are that your instance started booting and got as far as the boot loader launching the kernel, but at some point after that things went sideways. Now we start the iterative process of building disk images, turning them into AMIs, launching said AMIs, and seeing where they break. Some things you’ll probably run into here:
EC2 instances have two types of console available to them: A serial console and an VGA console. (Or rather, emulated serial and emulated VGA.) If you can have your kernel output go to both consoles, I recommend doing that. If you have to pick one, the serial console (which shows up as the “System Log” in EC2) is probably more useful than the VGA console (which shows up as “instance screenshot”) since it lets you see more than one screen of logs at once; but there’s a catch: Due to some bizarre breakage in EC2 — which I’ve been complaining about for ten years — the serial console is very “laggy”. If you find that you’re not getting any output, wait five minutes and try again.
You may need to tell your kernel where to find the root filesystem. On FreeBSD we build our disk images using GPT labels, so we simply need to specify in /etc/fstab that the root filesystem is on /dev/gpt/rootfs; but if you can’t do this, you’ll probably need to have different AMIs for Nitro instances vs. non-Nitro instances since Xen block devices will typically show up with different device names from NVMe disks. On FreeBSD, I also needed to set the vfs.root.mountfrom kernel environment variable for a while; this also is no longer needed on FreeBSD but something similar may be needed on other systems.
You’ll need to enable networking, using DHCP. On FreeBSD, this means placing ifconfigDEFAULT=“SYNCDHCP” into /etc/rc.conf; other systems will have other ways of specifying network parameters, and it may be necessary to specify a setting for the Xen network device, Intel SR-IOV network, and the Amazon ENA interface so that you’ll have the necessary configuration across all EC2 instance types. (On FreeBSD, ifconfigDEFAULT takes care of specifying the network settings which should apply for whatever network interface the kernel finds at boot time.)
You’ll almost certainly want to turn on SSH, so that you can connect into newly launched instances and make use of them. Don’t worry about setting a password or creating a user to SSH into yet — we’ll take care of that later.
EC2 configuration
Now it’s time to make the AMI behave like an EC2 instance. To this end, I prepared a set of rc.d scripts for FreeBSD. Most importantly, they
Print the SSH host keys to the console, so that you can veriy that they are correct when you first SSH in. (Remember, Verifying SSH host keys is more important than flossing every day.)
Download the SSH public key you want to use for logging in, and create an account (by default, “ec2-user”) with that key set up for you.
Fetch EC2 user-data and process it via configinit to allow you to configure the system as part of the process of launching it.
If your OS has an rc system derived from NetBSD’s rc.d, you may be able to use these scripts without any changes by simply installing them and enabling them in /etc/rc.conf; otherwise you may need to write your own scripts using mine as a model.
Firstboot scripts
A feature I added to FreeBSD a few years ago is the concept of “firstboot” scripts: These startup scripts are only run the first time a system boots. The aforementioned configinit and SSH key fetching scripts are flagged this way — so if your OS doesn’t support the “firstboot” keyword on rc.d scripts you’ll need to hack around that — but EC2 instances also ship with other scripts set to run on the first boot:
FreeBSD Update will fetch and install security and critical errata updates, and then reboot the system if necessary.
The UFS filesystem on the “boot disk” will be automatically expanded to the full size of the disk — this makes it possible to specify a larger size of disk at EC2 instance launch time.
Third-party packages will be automatically fetched and installed, according to a list in /etc/rc.conf. This is most useful if configinit is used to edit /etc/rc.conf, since it allows you to specify packages to install via the EC2 user-data.
While none of these are strictly necessary, I find them to be extremely useful and highly recommend implementing similar functionality in your systems.
Support my work!
I hope you find this useful, or at very least interesting. Please consider supporting my work in this area; while I’m happy to contribute my time to supporting open source software, it would be nice if I had money coming in which I could use to cover incidental expenses (e.g., conference travel) so that I didn’t end up paying to contribute to FreeBSD.

Digital Ocean
https://do.co/bsdnow

###Traceability, by Vint Cerf

  • A recent article from the August issue of the Communications of the ACM, for your contemplation:

At a recent workshop on cybersecurity in the U.K., a primary topic of consideration was how to preserve the freedom and openness of the Internet while protecting against the harmful behaviors that have emerged in this global medium. That this is a significant challenge cannot be overstated. The bad behaviors range from social network bullying and misinformation to email spam, distributed denial of service attacks, direct cyberattacks against infrastructure, malware propagation, identity theft, and a host of other ills requiring a wide range of technical and legal considerations. That these harmful behaviors can and do cross international boundaries only makes it more difficult to fashion effective responses.
In other columns, I have argued for better software development tools to reduce the common mistakes that lead to vulnerabilities that are exploited. Here, I want to focus on another aspect of response related to law enforcement and tracking down perpetrators. Of course, not all harms are (or perhaps are not yet) illegal, but discovering those who cause them may still be warranted. The recent adoption and implementation of the General Data Protection Regulation (GDPR) in the European Union creates an interesting tension because it highlights the importance and value of privacy while those who do direct or indirect harm must be tracked down and their identities discovered.
In passing, I mention that cryptography has sometimes been blamed for protecting the identity or actions of criminals but it is also a tool for protecting privacy. Arguments have been made for “back doors” to cryptographic systems but I am of the opinion that such proposals carry extremely high risk to privacy and safety. It is not my intent to argue this question in this column.
What is of interest to me is a concept to which I was introduced at the Ditchley workshop, specifically, differential traceability. The ability to trace bad actors to bring them to justice seems to me an important goal in a civilized society. The tension with privacy protection leads to the idea that only under appropriate conditions can privacy be violated. By way of example, consider license plates on cars. They are usually arbitrary identifiers and special authority is needed to match them with the car owners (unless, of course, they are vanity plates like mine: “Cerfsup”). This is an example of differential traceability; the police department has the authority to demand ownership information from the Department of Motor Vehicles that issues the license plates. Ordinary citizens do not have this authority.
In the Internet environment there are a variety of identifiers associated with users (including corporate users). Domain names, IP addresses, email addresses, and public cryptography keys are examples among many others. Some of these identifiers are dynamic and thus ambiguous. For example, IP addresses are not always permanent and may change (for example, temporary IP addresses assigned at Wi-Fi hotspots) or may be ambiguous in the case of Network Address Translation. Information about the time of assignment and the party to whom an IP address was assigned may be needed to identify an individual user. There has been considerable debate and even a recent court case regarding requirements to register users in domain name WHOIS databases in the context of the adoption of GDPR. If we are to accomplish the simultaneous objectives of protecting privacy while apprehending those engaged in harmful or criminal behavior on the Internet, we must find some balance between conflicting but desirable outcomes.
This suggests to me that the notion of traceability under (internationally?) agreed circumstances (that is, differential traceability) might be a fruitful concept to explore. In most societies today, it is accepted that we must be identifiable to appropriate authorities under certain conditions (consider border crossings, traffic violation stops as examples). While there are conditions under which apparent anonymity is desirable and even justifiable (whistle-blowing, for example) absolute anonymity is actually quite difficult to achieve (another point made at the Ditchley workshop) and might not be absolutely desirable given the misbehaviors apparent anonymity invites. I expect this is a controversial conclusion and I look forward to subsequent discussion.

###Remote Access Console using FreeBSD on an RPi3

  • Our friend, and FOSDEM Booth Neighbour, Jorge, has posted a tutorial on how he created a remote access console for his SmartOS server and other machines in his homelab
  • Parts:
  • Raspberry Pi 3 B+
  • NavoLabs micro POE Hat
  • FT4232H based USB-to-RS232 (4x) adapter
  • Official Raspberry Pi case (optional)
  • Heat-sink kit (optional)
  • USB-to-TTL adaptor (optional)
  • Sandisk 16Gb microSD

For the software I ended up using conserver. Below is a very brief tutorial on how to set everything up. I assume you have basic unix skills.

  • Get an RPi3 image, make some minor modifications for RPi3+, and write it to the USB stick
  • Configure FreeBSD on the RPi3
    • Load the ‘muge’ Ethernet Driver
    • Load USB serial support
    • Load the FTDI driver
    • Enable SSHd and Conserver
    • Configure Conserver
    • Setup log rotation
    • Start Conserver
  • And you’re good to go

A small bonus script I wrote to turn on the 2nd LED on the rPI once the system is booted, it will then blink the LED if someone is connected to any of the consoles.

##Beastie Bits

Tarsnap

##Feedback/Questions
We need more feedback emails. Please write to feedback@bsdnow.tv

Additionally, we are considering a new segment to be added to the end of the show (to make it skippable), where we have a ~15 minute deep dive on a topic. Some initial ideas are on the Virtual Memory subsystem, the Scheduler, Capsicum, and GEOM. What topics would you like to get very detailed explanations of? Many of the explanations may have accompanying graphics, and not be very suitable for audio only listeners, that is why we are planning to put it at the very end of the episode.

  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Thu, 23 Aug 2018 02:00:00 -0700
Episode 259: Long Live Unix | BSD Now 259
The strange birth and long life of Unix, FreeBSD jail with a single public IP, EuroBSDcon 2018 talks and schedule, OpenBSD on G4 iBook, PAM template user, ZFS file server, and reflections on one year of OpenBSD use. Picking the contest winner
  • Vincent
  • Bostjan
  • Andrew
  • Klaus-Hendrik
  • Will
  • Toby
  • Johnny
  • David
  • manfrom
  • Niclas
  • Gary
  • Eddy
  • Bruce
  • Lizz
  • Jim
  • Random number generator

    ##Headlines
    ###The Strange Birth and Long Life of Unix

    They say that when one door closes on you, another opens. People generally offer this bit of wisdom just to lend some solace after a misfortune. But sometimes it’s actually true. It certainly was for Ken Thompson and the late Dennis Ritchie, two of the greats of 20th-century information technology, when they created the Unix operating system, now considered one of the most inspiring and influential pieces of software ever written.
    A door had slammed shut for Thompson and Ritchie in March of 1969, when their employer, the American Telephone & Telegraph Co., withdrew from a collaborative project with the Massachusetts Institute of Technology and General Electric to create an interactive time-sharing system called Multics, which stood for “Multiplexed Information and Computing Service.” Time-sharing, a technique that lets multiple people use a single computer simultaneously, had been invented only a decade earlier. Multics was to combine time-sharing with other technological advances of the era, allowing users to phone a computer from remote terminals and then read e-mail, edit documents, run calculations, and so forth. It was to be a great leap forward from the way computers were mostly being used, with people tediously preparing and submitting batch jobs on punch cards to be run one by one.
    Over five years, AT&T invested millions in the Multics project, purchasing a GE-645 mainframe computer and dedicating to the effort many of the top researchers at the company’s renowned Bell Telephone Laboratories—­including Thompson and Ritchie, Joseph F. Ossanna, Stuart Feldman, M. Douglas McIlroy, and the late Robert Morris. But the new system was too ambitious, and it fell troublingly behind schedule. In the end, AT&T’s corporate leaders decided to pull the plug.
    After AT&T’s departure from the Multics project, managers at Bell Labs, in Murray Hill, N.J., became reluctant to allow any further work on computer operating systems, leaving some researchers there very frustrated. Although Multics hadn’t met many of its objectives, it had, as Ritchie later recalled, provided them with a “convenient interactive computing service, a good environment in which to do programming, [and] a system around which a fellowship could form.” Suddenly, it was gone.
    With heavy hearts, the researchers returned to using their old batch system. At such an inauspicious moment, with management dead set against the idea, it surely would have seemed foolhardy to continue designing computer operating systems. But that’s exactly what Thompson, Ritchie, and many of their Bell Labs colleagues did. Now, some 40 years later, we should be thankful that these programmers ignored their bosses and continued their labor of love, which gave the world Unix, one of the greatest computer operating systems of all time.
    The rogue project began in earnest when Thompson, Ritchie, and a third Bell Labs colleague, Rudd Canaday, began to sketch out on paper the design for a file system. Thompson then wrote the basics of a new operating system for the lab’s GE-645 mainframe. But with the Multics project ended, so too was the need for the GE-645. Thompson realized that any further programming he did on it was likely to go nowhere, so he dropped the effort.
    Thompson had passed some of his time after the demise of Multics writing a computer game called Space Travel, which simulated all the major bodies in the solar system along with a spaceship that could fly around them. Written for the GE-645, Space Travel was clunky to play—and expensive: roughly US $75 a game for the CPU time. Hunting around, Thompson came across a dusty PDP-7, a minicomputer built by Digital Equipment Corp. that some of his Bell Labs colleagues had purchased earlier for a circuit-analysis project. Thompson rewrote Space Travel to run on it.
    And with that little programming exercise, a second door cracked ajar. It was to swing wide open during the summer of 1969 when Thompson’s wife, Bonnie, spent a month visiting his parents to show off their newborn son. Thompson took advantage of his temporary bachelor existence to write a good chunk of what would become the Unix operating system for the discarded PDP‑7. The name Unix stems from a joke one of Thompson’s colleagues made: Because the new operating system supported only one user (Thompson), he saw it as an emasculated version of Multics and dubbed it “Un-multiplexed Information and Computing Service,” or Unics. The name later morphed into Unix.
    Initially, Thompson used the GE-645 to compose and compile the software, which he then downloaded to the PDP‑7. But he soon weaned himself from the mainframe, and by the end of 1969 he was able to write operating-system code on the PDP-7 itself. That was a step in the right direction. But Thompson and the others helping him knew that the PDP‑7, which was already obsolete, would not be able to sustain their skunkworks for long. They also knew that the lab’s management wasn’t about to allow any more research on operating systems.
    So Thompson and Ritchie got crea­tive. They formulated a proposal to their bosses to buy one of DEC’s newer minicomputers, a PDP-11, but couched the request in especially palatable terms. They said they were aiming to create tools for editing and formatting text, what you might call a word-processing system today. The fact that they would also have to write an operating system for the new machine to support the editor and text formatter was almost a footnote.
    Management took the bait, and an order for a PDP-11 was placed in May 1970. The machine itself arrived soon after, although the disk drives for it took more than six months to appear. During the interim, Thompson, Ritchie, and others continued to develop Unix on the PDP-7. After the PDP-11’s disks were installed, the researchers moved their increasingly complex operating system over to the new machine. Next they brought over the roff text formatter written by Ossanna and derived from the runoff program, which had been used in an earlier time-sharing system.
    Unix was put to its first real-world test within Bell Labs when three typists from AT&T’s patents department began using it to write, edit, and format patent applications. It was a hit. The patent department adopted the system wholeheartedly, which gave the researchers enough credibility to convince management to purchase another machine—a newer and more powerful PDP-11 model—allowing their stealth work on Unix to continue.
    During its earliest days, Unix evolved constantly, so the idea of issuing named versions or releases seemed inappropriate. But the researchers did issue new editions of the programmer’s manual periodically, and the early Unix systems were named after each such edition. The first edition of the manual was completed in November 1971.
    So what did the first edition of Unix offer that made it so great? For one thing, the system provided a hierarchical file system, which allowed something we all now take for granted: Files could be placed in directories—or equivalently, folders—that in turn could be put within other directories. Each file could contain no more than 64 kilobytes, and its name could be no more than six characters long. These restrictions seem awkwardly limiting now, but at the time they appeared perfectly adequate.
    Although Unix was ostensibly created for word processing, the only editor available in 1971 was the line-oriented ed. Today, ed is still the only editor guaranteed to be present on all Unix systems. Apart from the text-processing and general system applications, the first edition of Unix included games such as blackjack, chess, and tic-tac-toe. For the system administrator, there were tools to dump and restore disk images to magnetic tape, to read and write paper tapes, and to create, check, mount, and unmount removable disk packs.
    Most important, the system offered an interactive environment that by this time allowed time-sharing, so several people could use a single machine at once. Various programming languages were available to them, including BASIC, Fortran, the scripting of Unix commands, assembly language, and B. The last of these, a descendant of a BCPL (Basic Combined Programming Language), ultimately evolved into the immensely popular C language, which Ritchie created while also working on Unix.
    The first edition of Unix let programmers call 34 different low-level routines built into the operating system. It’s a testament to the system’s enduring nature that nearly all of these system calls are still available—and still heavily used—on modern Unix and Linux systems four decades on. For its time, first-­edition Unix provided a remarkably powerful environment for software development. Yet it contained just 4200 lines of code at its heart and occupied a measly 16 KB of main memory when it ran.
    Unix’s great influence can be traced in part to its elegant design, simplicity, portability, and serendipitous timing. But perhaps even more important was the devoted user community that soon grew up around it. And that came about only by an accident of its unique history.
    The story goes like this: For years Unix remained nothing more than a Bell Labs research project, but by 1973 its authors felt the system was mature enough for them to present a paper on its design and implementation at a symposium of the Association for Computing Machinery. That paper was published in 1974 in the Communications of the ACM. Its appearance brought a flurry of requests for copies of the software.
    This put AT&T in a bind. In 1956, AT&T had agreed to a U.S government consent decree that prevented the company from selling products not directly related to telephones and telecommunications, in return for its legal monopoly status in running the country’s long-distance phone service. So Unix could not be sold as a product. Instead, AT&T released the Unix source code under license to anyone who asked, charging only a nominal fee. The critical wrinkle here was that the consent decree prevented AT&T from supporting Unix. Indeed, for many years Bell Labs researchers proudly displayed their Unix policy at conferences with a slide that read, “No advertising, no support, no bug fixes, payment in advance.”
    With no other channels of support available to them, early Unix adopters banded together for mutual assistance, forming a loose network of user groups all over the world. They had the source code, which helped. And they didn’t view Unix as a standard software product, because nobody seemed to be looking after it. So these early Unix users themselves set about fixing bugs, writing new tools, and generally improving the system as they saw fit.
    The Usenix user group acted as a clearinghouse for the exchange of Unix software in the United States. People could send in magnetic tapes with new software or fixes to the system and get back tapes with the software and fixes that Usenix had received from others. In Australia, the University of New South Wales and the University of Sydney produced a more robust version of Unix, the Australian Unix Share Accounting Method, which could cope with larger numbers of concurrent users and offered better performance.
    By the mid-1970s, the environment of sharing that had sprung up around Unix resembled the open-source movement so prevalent today. Users far and wide were enthusiastically enhancing the system, and many of their improvements were being fed back to Bell Labs for incorporation in future releases. But as Unix became more popular, AT&T’s lawyers began looking harder at what various licensees were doing with their systems.
    One person who caught their eye was John Lions, a computer scientist then teaching at the University of New South Wales, in Australia. In 1977, he published what was probably the most famous computing book of the time, A Commentary on the Unix Operating System, which contained an annotated listing of the central source code for Unix.
    Unix’s licensing conditions allowed for the exchange of source code, and initially, Lions’s book was sold to licensees. But by 1979, AT&T’s lawyers had clamped down on the book’s distribution and use in academic classes. The anti­authoritarian Unix community reacted as you might expect, and samizdat copies of the book spread like wildfire. Many of us have nearly unreadable nth-­generation photocopies of the original book.
    End runs around AT&T’s lawyers indeed became the norm—even at Bell Labs. For example, between the release of the sixth edition of Unix in 1975 and the seventh edition in 1979, Thompson collected dozens of important bug fixes to the system, coming both from within and outside of Bell Labs. He wanted these to filter out to the existing Unix user base, but the company’s lawyers felt that this would constitute a form of support and balked at their release. Nevertheless, those bug fixes soon became widely distributed through unofficial channels. For instance, Lou Katz, the founding president of Usenix, received a phone call one day telling him that if he went down to a certain spot on Mountain Avenue (where Bell Labs was located) at 2 p.m., he would find something of interest. Sure enough, Katz found a magnetic tape with the bug fixes, which were rapidly in the hands of countless users.
    By the end of the 1970s, Unix, which had started a decade earlier as a reaction against the loss of a comfortable programming environment, was growing like a weed throughout academia and the IT industry. Unix would flower in the early 1980s before reaching the height of its popularity in the early 1990s.
    For many reasons, Unix has since given way to other commercial and noncommercial systems. But its legacy, that of an elegant, well-designed, comfortable environment for software development, lives on. In recognition of their accomplishment, Thompson and Ritchie were given the Japan Prize earlier this year, adding to a collection of honors that includes the United States’ National Medal of Technology and Innovation and the Association of Computing Machinery’s Turing Award. Many other, often very personal, tributes to Ritchie and his enormous influence on computing were widely shared after his death this past October.
    Unix is indeed one of the most influential operating systems ever invented. Its direct descendants now number in the hundreds. On one side of the family tree are various versions of Unix proper, which began to be commercialized in the 1980s after the Bell System monopoly was broken up, freeing AT&T from the stipulations of the 1956 consent decree. On the other side are various Unix-like operating systems derived from the version of Unix developed at the University of California, Berkeley, including the one Apple uses today on its computers, OS X. I say “Unix-like” because the developers of the Berkeley Software Distribution (BSD) Unix on which these systems were based worked hard to remove all the original AT&T code so that their software and its descendants would be freely distributable.
    The effectiveness of those efforts were, however, called into question when the AT&T subsidiary Unix System Laboratories filed suit against Berkeley Software Design and the Regents of the University of California in 1992 over intellectual property rights to this software. The university in turn filed a counterclaim against AT&T for breaches to the license it provided AT&T for the use of code developed at Berkeley. The ensuing legal quagmire slowed the development of free Unix-like clones, including 386BSD, which was designed for the Intel 386 chip, the CPU then found in many IBM PCs.
    Had this operating system been available at the time, Linus Torvalds says he probably wouldn’t have created Linux, an open-source Unix-like operating system he developed from scratch for PCs in the early 1990s. Linux has carried the Unix baton forward into the 21st century, powering a wide range of digital gadgets including wireless routers, televisions, desktop PCs, and Android smartphones. It even runs some supercomputers.
    Although AT&T quickly settled its legal disputes with Berkeley Software Design and the University of California, legal wrangling over intellectual property claims to various parts of Unix and Linux have continued over the years, often involving byzantine corporate relations. By 2004, no fewer than five major lawsuits had been filed. Just this past August, a software company called the TSG Group (formerly known as the SCO Group), lost a bid in court to claim ownership of Unix copyrights that Novell had acquired when it purchased the Unix System Laboratories from AT&T in 1993.
    As a programmer and Unix historian, I can’t help but find all this legal sparring a bit sad. From the very start, the authors and users of Unix worked as best they could to build and share, even if that meant defying authority. That outpouring of selflessness stands in sharp contrast to the greed that has driven subsequent legal battles over the ownership of Unix.
    The world of computer hardware and software moves forward startlingly fast. For IT professionals, the rapid pace of change is typically a wonderful thing. But it makes us susceptible to the loss of our own history, including important lessons from the past. To address this issue in a small way, in 1995 I started a mailing list of old-time Unix ­aficionados. That effort morphed into the Unix Heritage Society. Our goal is not only to save the history of Unix but also to collect and curate these old systems and, where possible, bring them back to life. With help from many talented members of this society, I was able to restore much of the old Unix software to working order, including Ritchie’s first C compiler from 1972 and the first Unix system to be written in C, dating from 1973.
    One holy grail that eluded us for a long time was the first edition of Unix in any form, electronic or otherwise. Then, in 2006, Al Kossow from the Computer History Museum, in Mountain View, Calif., unearthed a printed study of Unix dated 1972, which not only covered the internal workings of Unix but also included a complete assembly listing of the kernel, the main component of this operating system. This was an amazing find—like discovering an old Ford Model T collecting dust in a corner of a barn. But we didn’t just want to admire the chrome work from afar. We wanted to see the thing run again.
    In 2008, Tim Newsham, an independent programmer in Hawaii, and I assembled a team of like-minded Unix enthusiasts and set out to bring this ancient system back from the dead. The work was technically arduous and often frustrating, but in the end, we had a copy of the first edition of Unix running on an emulated PDP-11/20. We sent out messages announcing our success to all those we thought would be interested. Thompson, always succinct, simply replied, “Amazing.” Indeed, his brainchild was amazing, and I’ve been happy to do what I can to make it, and the story behind it, better known.

    Digital Ocean
    http://do.co/bsdnow

    ###FreeBSD jails with a single public IP address

    Jails in FreeBSD provide a simple yet flexible way to set up a proper server layout. In the most setups the actual server only acts as the host system for the jails while the applications themselves run within those independent containers. Traditionally every jail has it’s own IP for the user to be able to address the individual services. But if you’re still using IPv4 this might get you in trouble as the most hosters don’t offer more than one single public IP address per server.

    • Create the internal network

    In this case NAT (“Network Address Translation”) is a good way to expose services in different jails using the same IP address.
    First, let’s create an internal network (“NAT network”) at 192.168.0.0/24. You could generally use any private IPv4 address space as specified in RFC 1918. Here’s an overview: https://en.wikipedia.org/wiki/Privatenetwork. Using pf, FreeBSD’s firewall, we will map requests on different ports of the same public IP address to our individual jails as well as provide network access to the jails themselves.
    First let’s check which network devices are available. In my case there’s em0 which provides connectivity to the internet and lo0, the local loopback device.

    options=209b<RXCSUM,TXCSUM,VLANMTU,VLANHWTAGGING,VLANHWCSUM,WOLMAGIC> [...] inet 172.31.1.100 netmask 0xffffff00 broadcast 172.31.1.255 nd6 options=23<PERFORMNUD,ACCEPTRTADV,AUTO_LINKLOCAL> media: Ethernet autoselect (1000baseT <full-duplex>) status: active lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUMIPV6,TXCSUMIPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>``` > For our internal network, we create a cloned loopback device called lo1. Therefore we need to customize the /etc/rc.conf file, adding the following two lines: cloned_interfaces="lo1" ipv4_addrs_lo1="192.168.0.1-9/29" > This defines a /29 network, offering IP addresses for a maximum of 6 jails: ipcalc 192.168.0.1/29 Address: 192.168.0.1 11000000.10101000.00000000.00000 001 Netmask: 255.255.255.248 = 29 11111111.11111111.11111111.11111 000 Wildcard: 0.0.0.7 00000000.00000000.00000000.00000 111 => Network: 192.168.0.0/29 11000000.10101000.00000000.00000 000 HostMin: 192.168.0.1 11000000.10101000.00000000.00000 001 HostMax: 192.168.0.6 11000000.10101000.00000000.00000 110 Broadcast: 192.168.0.7 11000000.10101000.00000000.00000 111 Hosts/Net: 6 Class C, Private Internet > Then we need to restart the network. Please be aware of currently active SSH sessions as they might be dropped during restart. It’s a good moment to ensure you have KVM access to that server ;-) service netif restart > After reconnecting, our newly created loopback device is active: lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> inet 192.168.0.1 netmask 0xfffffff8 inet 192.168.0.2 netmask 0xffffffff inet 192.168.0.3 netmask 0xffffffff inet 192.168.0.4 netmask 0xffffffff inet 192.168.0.5 netmask 0xffffffff inet 192.168.0.6 netmask 0xffffffff inet 192.168.0.7 netmask 0xffffffff inet 192.168.0.8 netmask 0xffffffff inet 192.168.0.9 netmask 0xffffffff nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> Setting up > pf part of the FreeBSD base system, so we only have to configure and enable it. By this moment you should already have a clue of which services you want to expose. If this is not the case, just fix that file later on. In my example configuration, I have a jail running a webserver and another jail running a mailserver: Public IP address IP_PUB="1.2.3.4" Packet normalization scrub in all Allow outbound connections from within the jails nat on em0 from lo1:network to any -> (em0) webserver jail at 192.168.0.2 rdr on em0 proto tcp from any to $IP_PUB port 443 -> 192.168.0.2 just an example in case you want to redirect to another port within your jail rdr on em0 proto tcp from any to $IP_PUB port 80 -> 192.168.0.2 port 8080 mailserver jail at 192.168.0.3 rdr on em0 proto tcp from any to $IP_PUB port 25 -> 192.168.0.3 rdr on em0 proto tcp from any to $IP_PUB port 587 -> 192.168.0.3 rdr on em0 proto tcp from any to $IP_PUB port 143 -> 192.168.0.3 rdr on em0 proto tcp from any to $IP_PUB port 993 -> 192.168.0.3 > Now just enable pf like this (which is the equivalent of adding pf_enable=YES to /etc/rc.conf): sysrc pf_enable="YES" > and start it: service pf start Install ezjail > Ezjail is a collection of scripts by erdgeist that allow you to easily manage your jails. pkg install ezjail > As an alternative, you could install ezjail from the ports tree. Now we need to set up the basejail which contains the shared base system for our jails. In fact, every jail that you create get’s will use that basejail to symlink directories related to the base system like /bin and /sbin. This can be accomplished by running ezjail-admin install > In the next step, we’ll copy the /etc/resolv.conf file from our host to the newjail, which is the template for newly created jails (the parts that are not provided by basejail), to ensure that domain resolution will work properly within our jails later on: cp /etc/resolv.conf /usr/jails/newjail/etc/ > Last but not least, we enable ezjail and start it: sysrc ezjail_enable="YES" service ezjail start Create a jail > Creating a jail is as easy as it could probably be: ezjail-admin create webserver 192.168.0.2 ezjail-admin start webserver > Now you can access your jail using: ezjail-admin console webserver > Each jail contains a vanilla FreeBSD installation. Deploy services > Now you can spin up as many jails as you want to set up your services like web, mail or file shares. You should take care not to enable sshd within your jails, because that would cause problems with the service’s IP bindings. But this is not a problem, just SSH to the host and enter your jail using ezjail-admin console. EuroBSDcon 2018 Talks & Schedule (https://2018.eurobsdcon.org/talks-schedule/) News Roundup OpenBSD on an iBook G4 (https://bobstechsite.com/openbsd-on-an-ibook-g4/) > I've mentioned on social media and on the BTS podcast a few times that I wanted to try installing OpenBSD onto an old "snow white" iBook G4 I acquired last summer to see if I could make it a useful machine again in the year 2018. This particular eBay purchase came with a 14" 1024x768 TFT screen, 1.07GHz PowerPC G4 processor, 1.5GB RAM, 100GB of HDD space and an ATI Radeon 9200 graphics card with 32 MB of SDRAM. The optical drive, ethernet port, battery & USB slots are also fully-functional. The only thing that doesn't work is the CMOS battery, but that's not unexpected for a device that was originally released in 2004. Initial experiments > This iBook originally arrived at my door running Apple Mac OSX Leopard and came with the original install disk, the iLife & iWork suites for 2008, various instruction manuals, a working power cable and a spare keyboard. As you'll see in the pictures I took for this post the characters on the buttons have started to wear away from 14 years of intensive use, but the replacement needs a very good clean before I decide to swap it in! > After spending some time exploring the last version of OSX to support the IBM PowerPC processor architecture I tried to see if the hardware was capable of modern computing with Linux. Something I knew ahead of trying this was that the WiFi adapter was unlikely to work because it's a highly proprietary component designed by Apple to work specifically with OSX and nothing else, but I figured I could probably use a wireless USB dongle later to get around this limitation. > Unfortunately I found that no recent versions of mainstream Linux distributions would boot off this machine. Debian has dropped support 32-bit PowerPC architectures and the PowerPC variants of Ubuntu 16.04 LTS (vanilla, MATE and Lubuntu) wouldn't even boot the installer! The only distribution I could reliably install on the hardware was Lubuntu 14.04 LTS. > Unfortunately I'm not the biggest fan of the LXDE desktop for regular work and a lot of ported applications were old and broken because it clearly wasn't being maintained by people that use the hardware anymore. Ubuntu 14.04 is also approaching the end of its support life in early 2019, so this limited solution also has a limited shelf-life. Over to BSD > I discussed this problem with a few people on Mastodon and it was pointed out to me that OSX is built on the Darwin kernel, which happens to be a variant of BSD. NetBSD and OpenBSD fans in particular convinced me that their communities still saw the value of supporting these old pieces of kit and that I should give BSD a try. > So yesterday evening I finally downloaded the "macppc" version of OpenBSD 6.3 with no idea what to expect. I hoped for the best but feared the worst because my last experience with this operating system was trying out PC-BSD in 2008 and discovering with disappointment that it didn't support any of the hardware on my Toshiba laptop. > When I initially booted OpenBSD I was a little surprised to find the login screen provided no visual feedback when I typed in my password, but I can understand the security reasons for doing that. The initial desktop environment that was loaded was very basic. All I could see was a console output window, a terminal and a desktop switcher in the X11 environment the system had loaded. > After a little Googling I found this blog post had some fantastic instructions to follow for the post-installation steps: https://sohcahtoa.org.uk/openbsd.html. I did have to adjust them slightly though because my iBook only has 1.5GB RAM and not every package that page suggests is available on macppc by default. You can see a full list here: https://ftp.openbsd.org/pub/OpenBSD/6.3/packages/powerpc/. Final thoughts > I was really impressed with the performance of OpenBSD's "macppc" port. It boots much faster than OSX Leopard on the same hardware and unlike Lubuntu 14.04 it doesn't randomly hang for no reason or crash if you launch something demanding like the GIMP. > I was pleased to see that the command line tools I'm used to using on Linux have been ported across too. OpenBSD also had no issues with me performing basic desktop tasks on XFCE like browsing the web with NetSurf, playing audio files with VLC and editing images with the GIMP. Limited gaming is also theoretically possible if you're willing to build them (or an emulator) from source with SDL support. > If I wanted to use this system for heavy duty work then I'd probably be inclined to run key applications like LibreOffice on a Raspberry Pi and then connect my iBook G4 to those using VNC or an SSH connection with X11 forwarding. BSD is UNIX after all, so using my ancient laptop as a dumb terminal should work reasonably well. > In summary I was impressed with OpenBSD and its ability to breathe new life into this old Apple Mac. I'm genuinely excited about the idea of trying BSD with other devices on my network such as an old Asus Eee PC 900 netbook and at least one of the many Raspberry Pi devices I use. Whether I go the whole hog and replace Fedora on my main production laptop though remains to be seen! The template user with PAM and login(1) (http://oshogbo.vexillium.org/blog/48) > When you build a new service (or an appliance) you need your users to be able to configure it from the command line. To accomplish this you can create system accounts for all registered users in your service and assign them a special login shell which provides such limited functionality. This can be painful if you have a dynamic user database. > Another challenge is authentication via remote services such as RADIUS. How can we implement services when we authenticate through it and log into it as a different user? Furthermore, imagine a scenario when RADIUS decides on which account we have the right to access by sending an additional attribute. > To address these two problems we can use a "template" user. Any of the PAM modules can set the value of the PAM_USER item. The value of this item will be used to determine which account we want to login. Only the "template" user must exist on the local password database, but the credential check can be omitted by the module. > This functionality exists in the login(1) used by FreeBSD, HardenedBSD, DragonFlyBSD and illumos. The functionality doesn't exist in the login(1) used in NetBSD, and OpenBSD doesn't support PAM modules at all. In addition what is also noteworthy is that such functionality was also in the OpenSSH but they decided to remove it and call it a security vulnerability (CVE 2015-6563). I can see how some people may have seen it that way, that’s why I recommend reading this article from an OpenPAM author and a FreeBSD security officer at the time. > Knowing the background let's take a look at an example. ```PAMEXTERN int pamsmauthenticate(pamhandlet *pamh, int flags _unused, int argc _unused, const char *argv[] _unused) { const char *user, *password; int err; err = pam_get_user(pamh, &user, NULL); if (err != PAM_SUCCESS) return (err); err = pam_get_authtok(pamh, PAM_AUTHTOK, &password, NULL); if (err == PAM_CONV_ERR) return (err); if (err != PAM_SUCCESS) return (PAM_AUTH_ERR); err = authenticate(user, password); if (err != PAM_SUCCESS) { return (err); } return (pam_set_item(pamh, PAM_USER, "template")); }

    In the listing above we have an example of a PAM module. The pamgetuser(3) provides a username. The pamgetauthtok(3) shows us a secret given by the user. Both functions allow us to give an optional prompt which should be shown to the user. The authenticate function is our crafted function which authenticates the user. In our first scenario we wanted to keep all users in an external database. If authentication is successful we then switch to a template user which has a shell set up for a script allowing us to configure the machine. In our second scenario the authenticate function authenticates the user in RADIUS.

    Another step is to add our PAM module to the /etc/pam.d/system or to the /etc/pam.d/login configuration:

    auth sufficient pamtemplate.so nowarn allowlocal

    Unfortunately the description of all these options goes beyond this article - if you would like to know more about it you can find them in the PAM manual. The last thing we need to do is to add our template user to the system which you can do by the adduser(8) command or just simply modifying the /etc/master.passwd file and use pwdmkdb(8) program:

    $ tail -n /etc/master.passwd
    template::1000:1000::0:0:User &:/:/usr/local/bin/templatesh
    $ sudo pwdmkdb /etc/master.passwd

    As you can see,the template user can be locked and we still can use it in our PAM module (the * character after login).
    I would like to thank Dag-Erling Smørgrav for pointing this functionality out to me when I was looking for it some time ago.

    iXsystems
    iXsystems @ VMWorld

    ###ZFS file server

    • What is the need?

    At work, we run a compute cluster that uses an Isilon cluster as primary NAS storage. Excluding snapshots, we have about 200TB of research data, some of them in compressed formats, and others not. We needed an offsite backup file server that would constantly mirror our primary NAS and serve as a quick recovery source in case of a data loss in the the primary NAS. This offsite file server would be passive - will never face the wrath of the primary cluster workload.
    In addition to the role of a passive backup server, this solution would take on some passive report generation workloads as an ideal way of offloading some work from the primary NAS. The passive work is read-only.
    The backup server would keep snapshots in a best effort basis dating back to 10 years. However, this data on this backup server would be archived to tapes periodically.

    • A simple guidance of priorities:

    • Data integrity > Cost of solution > Storage capacity > Performance.

    • Why not enterprise NAS? NetApp FAS or EMC Isilon or the like?

    We decided that enterprise grade NAS like NetAPP FAS or EMC Isilon are prohibitively expensive and an overkill for our needs.
    An open source & cheaper alternative to enterprise grade filesystem with the level of durability we expect turned up to be ZFS. We’re already spoilt from using snapshots by a clever Copy-on-Write Filesystem(WAFL) by NetApp. ZFS providing snapshots in almost identical way was a big influence in the choice. This is also why we did not consider just a CentOS box with the default XFS filesystem.

    • FreeBSD vs Debian for ZFS

    This is a backup server, a long-term solution. Stability and reliability are key requirements. ZFS on Linux may be popular at this time, but there is a lot of churn around its development, which means there is a higher probability of bugs like this to occur. We’re not looking for cutting edge features here. Perhaps, Linux would be considered in the future.

    • FreeBSD + ZFS

    We already utilize FreeBSD and OpenBSD for infrastructure services and we have nothing but praises for the stability that the BSDs have provided us. We’d gladly use FreeBSD and OpenBSD wherever possible.

    • Okay, ZFS, but why not FreeNAS?

    IMHO, FreeNAS provides a integrated GUI management tool over FreeBSD for a novice user to setup and configure FreeBSD, ZFS, Jails and many other features. But, this user facing abstraction adds an extra layer of complexity to maintain that is just not worth it in simpler use cases like ours. For someone that appreciates the commandline interface, and understands FreeBSD enough to administer it, plain FreeBSD + ZFS is simpler and more robust than FreeNAS.

    • Specifications
    • Lenovo SR630 Rackserver
    • 2 X Intel Xeon silver 4110 CPUs
    • 768 GB of DDR4 ECC 2666 MHz RAM
    • 4 port SAS card configured in passthrough mode(JBOD)
    • Intel network card with 10 Gb SFP+ ports
    • 128GB M.2 SSD for use as boot drive
    • 2 X HGST 4U60 JBOD
    • 120(2 X 60) X 10TB SAS disks

    ###Reflection on one-year usage of OpenBSD

    I have used OpenBSD for more than one year, and it is time to give a summary of the experience:

    • (1) What do I get from OpenBSD?

    a) A good UNIX tutorial. When I am curious about some UNIXcommands’ implementation, I will refer to OpenBSD source code, and I actually gain something every time. E.g., refresh socket programming skills from nc; know how to process file efficiently from cat.

    b) A better test bed. Although my work focus on developing programs on Linux, I will try to compile and run applications on OpenBSD if it is possible. One reason is OpenBSD usually gives more helpful warnings. E.g., hint like this:

    ......
    warning: sprintf() is often misused, please use snprintf()
    ......

    Or you can refer this post which I wrote before. The other is sometimes program run well on Linux may crash on OpenBSD, and OpenBSD can help you find hidden bugs.

    c) Some handy tools. E.g. I find tcpbench is useful, so I ported it into Linux for my own usage (project is here).

    • (2) What I give back to OpenBSD?

    a) Patches. Although most of them are trivial modifications, they are still my contributions.

    b) Write blog posts to share experience about using OpenBSD.

    c) Develop programs for OpenBSD/BSD: lscpu and free.

    d) Porting programs into OpenBSD: E.g., I find google/benchmark is a nifty tool, but lacks OpenBSD support, I submitted PR and it is accepted. So you can use google/benchmark on OpenBSD now.

    • Generally speaking, the time invested on OpenBSD is rewarding. If you are still hesitating, why not give a shot?

    ##Beastie Bits

    Tarsnap

    ##Feedback/Questions

    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Thu, 16 Aug 2018 00:00:00 -0700
    Episode 258: OS Foundations | BSD Now 258
    FreeBSD Foundation July Newsletter, a bunch of BSDCan trip reports, HardenedBSD Foundation status, FreeBSD and OSPFd, ZFS disk structure overview, and more Spectre mitigations in OpenBSD.

    ##Headlines
    FreeBSD Foundation Update, July 2018

    • MESSAGE FROM THE EXECUTIVE DIRECTOR

    We’re in the middle of summer here, in Boulder, CO. While the days are typically hot, they can also be quite unpredictable. Thanks to the Rocky Mountains, waking up to 50-degree (~10 C) foggy weather is not surprising. In spite of the unpredictable weather, many of us took some vacation this month. Whether it was extending the Fourth of July celebration, spending time with family, or relaxing and enjoying the summer weather, we appreciated our time off, while still managing to accomplish a lot!
    In this newsletter, Glen Barber enlightens us about the upcoming 12.0 release. I gave a recap of OSCON, that Ed Maste and I attended, and Mark Johnston explains the work on his improved microcode loading project, that we are funding. Finally, Anne Dickison gives us a rundown on upcoming events and information on submitting a talk for MeetBSD.
    Your support helps us continue this work. Please consider making a donation today. We can’t do it without you. Happy reading!!

    • June 2018 Development Projects Update
    • Fundraising Update: Supporting the Project
    • July 2018 Release Engineering Update
    • OSCON 2018 Recap
    • Submit Your Work: MeetBSD 2018
    • FreeBSD Discount for 2018 SNIA Developer Conference
    • EuroBSDcon 2018 Travel Grant Application Deadline: August 2

    iXsystems

    ###BSDCan Trip Reports

    ##News Roundup
    FreeBSD and OSPFd

    With FreeBSD jails deployed around the world, static routing was getting a bit out of hand. Plus, when I needed to move a jail from one data center to another, I would have to update routing tables across multiple sites. Not ideal. Enter dynamic routing…

    OSPF (open shortest path first) is an internal dynamic routing protocol that provides the autonomy that I needed and it’s fairly easy to setup. This article does not cover configuration of VPN links, ZFS, or Freebsd jails, however it’s recommended that you use seperate ZFS datasets per jail so that migration between hosts can be done with zfs send & receive.

    In this scenario, we have five FreeBSD servers in two different data centers. Each physical server runs anywhere between three to ten jails. When jails are deployed, they are assigned a /32 IP on lo2. From here, pf handles inbound port forwarding and outbound NAT. Links between each server are provided by OpenVPN TAP interfaces. (I used TAP to pass layer 2 traffic. I seem to remember that I needed TAP interfaces due to needing GRE tunnels on top of TUN interfaces to get OSPF to communicate. I’ve heard TAP is slower than TUN so I may revisit this.)

    In this example, we will use 172.16.2.0/24 as the range for OpenVPN P2P links and 172.16.3.0/24 as the range of IPs available for assignment to each jail. Previously, when deploying a jail, I assigned IPs based on the following groups:

    Server 1: 172.16.3.0/28
    Server 2: 172.16.3.16/28
    Server 3: 172.16.3.32/28
    Server 4: 172.16.3.48/28
    Server 5: 172.16.3.64/28

    When statically routing, this made routing tables a bit smaller and easier to manage. However, when I needed to migrate a jail to a new host, I had to add a new /32 to all routing tables. Now, with OSPF, this is no longer an issue, nor is it required.

    • To get started, first we install the Quagga package.

    • The two configuration files needed to get OSPFv2 running are /usr/local/etc/quagga/zebra.conf and /usr/local/etc/quagga/ospfd.conf.

    • Starting with zebra.conf, we’ll define the hostname and a management password.

    • Second, we will populate the ospfd.conf file.

    • To break this down:

    • service advanced-vty allows you to skip the en or enable command. Since I’m the only one who uses this service, it’s one less command to type.

    • ip ospf authentication message-digest and ip ospf message-diget-key… ignores non-authenticated OSPF communication. This is useful when communicating over the WAN and to prevent a replay attack. Since I’m using a VPN to communicate, I could exclude these.

    • passive-interface default turns off the active communication of OSPF messages on all interfaces except for the interfaces listed as no passive-interface [interface name]. Since my ospf communication needs to leverage the VPNs, this prevents the servers from trying to send ospf data out the WAN interface (a firewall would work too).

    • network 172.16.2.0/23 area 0.0.0.0 lists a supernet of both 172.16.2.0/24 and 172.16.3.0/24. This ensures routes for the jails are advertised along with the P2P links used by OpenVPN. The OpenVPN links are not required but can provide another IP to access your server if one of the links goes down. (See the suggested tasks below).

    • At this point, we can enable the services in rc.conf.local and start them.

    • We bind the management interface to 127.0.0.1 so that it’s only accessable to local telnet sessions. If you want to access this service remotely, you can bind to a remotely accessable IP. Remember telnet is not secure. If you need remote access, use a VPN.

    • To manage the services, you can telnet to your host’s localhost address.

    • Use 2604 for the ospf service.

    • Remember, this is accessible by non-root users so set a good password.

    ###A broad overview of how ZFS is structured on disk

    When I wrote yesterday’s entry, it became clear that I didn’t understand as much about how ZFS is structured on disk (and that this matters, since I thought that ZFS copy on write updates updated a lot more than they do). So today I want to write down my new broad understanding of how this works. (All of this can be dug out of the old, draft ZFS on-disk format specification, but that spec is written in a very detailed way and things aren’t always immediately clear from it.)

    Almost everything in ZFS is in DMU object. All objects are defined by a dnode, and object dnodes are almost always grouped together in an object set. Object sets are themselves DMU objects; they store dnodes as basically a giant array in a ‘file’, which uses data blocks and indirect blocks and so on, just like anything else. Within a single object set, dnodes have an object number, which is the index of their position in the object set’s array of dnodes. (Because an object number is just the index of the object’s dnode in its object set’s array of dnodes, object numbers are basically always going to be duplicated between object sets (and they’re always relative to an object set). For instance, pretty much every object set is going to have an object number ten, although not all object sets may have enough objects that they have an object number ten thousand. One corollary of this is that if you ask zdb to tell you about a given object number, you have to tell zdb what object set you’re talking about. Usually you do this by telling zdb which ZFS filesystem or dataset you mean.)

    Each ZFS filesystem has its own object set for objects (and thus dnodes) used in the filesystem. As I discovered yesterday, every ZFS filesystem has a directory hierarchy and it may go many levels deep, but all of this directory hierarchy refers to directories and files using their object number.

    ZFS organizes and keeps track of filesystems, clones, and snapshots through the DSL (Dataset and Snapshot Layer). The DSL has all sorts of things; DSL directories, DSL datasets, and so on, all of which are objects and many of which refer to object sets (for example, every ZFS filesystem must refer to its current object set somehow). All of these DSL objects are themselves stored as dnodes in another object set, the Meta Object Set, which the uberblock points to. To my surprise, object sets are not stored in the MOS (and as a result do not have ‘object numbers’). Object sets are always referred to directly, without indirection, using a block pointer to the object set’s dnode. (I think object sets are referred to directly so that snapshots can freeze their object set very simply.)

    The DSL directories and datasets for your pool’s set of filesystems form a tree themselves (each filesystem has a DSL directory and at least one DSL dataset). However, just like in ZFS filesystems, all of the objects in this second tree refer to each other indirectly, by their MOS object number. Just as with files in ZFS filesystems, this level of indirection limits the amount of copy on write updates that ZFS had to do when something changes.

    PS: If you want to examine MOS objects with zdb, I think you do it with something like ‘zdb -vvv -d ssddata 1’, which will get you object number 1 of the MOS, which is the MOS object directory. If you want to ask zdb about an object in the pool’s root filesystem, use ‘zdb -vvv -d ssddata/ 1’. You can tell which one you’re getting depending on what zdb prints out. If it says ‘Dataset mos [META]’ you’re looking at objects from the MOS; if it says ‘Dataset ssddata [ZPL]’, you’re looking at the pool’s root filesystem (where object number 1 is the ZFS master node).

    PPS: I was going to write up what changed on a filesystem write, but then I realized that I didn’t know how blocks being allocated and freed are reflected in pool structures. So I’ll just say that I think that ignoring free space management, only four DMU objects get updated; the file itself, the filesystem’s object set, the filesystem’s DSL dataset object, and the MOS.

    • (As usual, doing the research to write this up taught me things that I didn’t know about ZFS.)

    Digital Ocean

    ###HardenedBSD Foundation Status

    On 09 July 2018, the HardenedBSD Foundation Board of Directors held the kick-off meeting to start organizing the Foundation. The following people attended the kick-off meeting:

    • Shawn Webb (in person)
    • George Saylor (in person)
    • Ben Welch (in person)
    • Virginia Suydan (in person)
    • Ben La Monica (phone)
    • Dean Freeman (phone)
    • Christian Severt (phone)

    We discussed the very first steps that need to be taken to organize the HardenedBSD Foundation as a 501©(3) not-for-profit organization in the US. We determined we could file a 1023EZ instead of the full-blown 1023. This will help speed the process up drastically.

    • The steps are laid out as follows:
    • Register a Post Office Box (PO Box) (completed on 10 Jul 2018).
    • Register The HardenedBSD Foundation as a tax-exempt nonstock corporation in the state of Maryland (started on 10 Jul 2018, submitted on 18 Jul 2018, granted 20 Jul 2018).
    • Obtain a federal tax ID (obtained 20 Jul 2018).
    • Close the current bank account and create a new one using the federal tax ID (completed on 20 Jul 2018).
    • File the 1023EZ paperwork with the federal government (started on 20 Jul 2018).
    • Hire an attorney to help draft the organization bylaws.
    • Each of the steps must be done serially and in order.

    We added Christian Severt, who is on Emerald Onion’s Board of Directors, to the HardenedBSD Foundation Board of Directors as an advisor. He was foundational in getting Emerald Onion their 501©(3) tax-exempt, not-for-profit status and has really good insight. Additionally, he’s going to help HardenedBSD coordinate hosting services, figuring out the best deals for us.

    We promoted George Saylor to Vice President and changed Shawn Webb’s title to President and Director. This is to help resolve potential concerns both the state and federal agencies might have with an organization having only a single President role.

    We hope to be granted our 501©(3) status before the end of the year, though that may be subject to change. We are excited for the formation of the HardenedBSD Foundation, which will open up new opportunities not otherwise available to HardenedBSD.

    ###More mitigations against speculative execution vulnerabilities

    Philip Guenther (guenther@) and Bryan Steele (brynet@) have added more mitigations against speculative execution CPU vulnerabilities on the amd64 platform.

    CVSROOT: /cvs Module name: src Changes by: guenther@cvs.openbsd.org 2018/07/23 11:54:04 Modified files: sys/arch/amd64/amd64: locore.S sys/arch/amd64/include: asm.h cpufunc.h frameasm.h Log message: Do "Return stack refilling", based on the "Return stack underflow" discussion and its associated appendix at https://support.google.com/faqs/answer/7625886 This should address at least some cases of "SpectreRSB" and earlier Spectre variants; more commits to follow. The refilling is done in the enter-kernel-from-userspace and return-to-userspace-from-kernel paths, making sure to do it before unblocking interrupts so that a successive interrupt can't get the CPU to C code without doing this refill. Per the link above, it also does it immediately after mwait, apparently in case the low-power CPU states of idle-via-mwait flush the RSB. ok mlarkin@ deraadt@``` and: ```CVSROOT: /cvs Module name: src Changes by: guenther@cvs.openbsd.org 2018/07/23 20:42:25 Modified files: sys/arch/amd64/amd64: locore.S vector.S vmm_support.S sys/arch/amd64/include: asm.h cpufunc.h Log message: Also do RSB refilling when context switching, after vmexits, and when vmlaunch or vmresume fails. Follow the lead of clang and the intel recommendation and do an lfence after the pause in the speculation-stop path for retpoline, RSB refill, and meltover ASM bits. ok kettenis@ deraadt@``` "Mitigation G-2" for AMD processors: ```CVSROOT: /cvs Module name: src Changes by: brynet@cvs.openbsd.org 2018/07/23 17:25:03 Modified files: sys/arch/amd64/amd64: identcpu.c sys/arch/amd64/include: specialreg.h Log message: Add "Mitigation G-2" per AMD's Whitepaper "Software Techniques for Managing Speculation on AMD Processors" By setting MSR C001_1029[1]=1, LFENCE becomes a dispatch serializing instruction. Tested on AMD FX-4100 "Bulldozer", and Linux guest in SVM vmd(8) ok deraadt@ mlarkin@``` Beastie Bits HardenedBSD will stop supporting 10-STABLE on 10 August 2018 (https://groups.google.com/a/hardenedbsd.org/forum/#!topic/users/xvU0g-g1l5U) GSoC 2018 Reports: Integrate libFuzzer with the Basesystem, Part 2 (https://blog.netbsd.org/tnf/entry/gsoc_2018_reports_integrate_libfuzzer1) ZFS Boot Environments at PBUG (https://vermaden.wordpress.com/2018/07/30/zfs-boot-environments-at-pbug/) Second Editions versus the Publishing Business (https://blather.michaelwlucas.com/archives/3229) Theo de Raadt on "unveil(2) usage in base" (https://undeadly.org/cgi?action=article;sid=20180728063716) rtadvd(8) has been replaced by rad(8) (https://undeadly.org/cgi?action=article;sid=20180724072205) BSD Users Stockholm Meetup #3 (https://www.meetup.com/BSD-Users-Stockholm/events/253447019/) Changes to NetBSD release support policy (https://blog.netbsd.org/tnf/entry/changes_to_netbsd_release_support) The future of HAMMER1 (http://lists.dragonflybsd.org/pipermail/users/2018-July/357832.html) *** Tarsnap Feedback/Questions Rodriguez - A Question (http://dpaste.com/0Y1B75Q#wrap) Shane - About ZFS Mostly (http://dpaste.com/32YGNBY#wrap) Leif - ZFS less than 8gb (http://dpaste.com/2GY6HHC#wrap) Wayne - ZFS vs EMC (http://dpaste.com/17PSCXC#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
    Tue, 07 Aug 2018 22:00:00 -0700
    Episode 257: Great NetBSD 8 | BSD Now 257
    NetBSD 8.0 available, FreeBSD on Scaleway’s ARM64 VPS, encrypted backups with OpenBSD, Dragonfly server storage upgrade, zpool checkpoints, g2k18 hackathon reports, and more.

    ##Headlines
    NetBSD v8.0 Released

    The NetBSD Project is pleased to announce NetBSD 8.0, the sixteenth major release of the NetBSD operating system.

    This release brings stability improvements, hundreds of bug fixes, and many new features.

    • Some highlights of the NetBSD 8.0 release are:

    • USB stack rework, USB3 support added.

    • In-kernel audio mixer (audio_system(9)).

    • Reproducible builds (MKREPRO, see mk.conf(5)).

    • Full userland debug information (MKDEBUG, see mk.conf(5)) available. While most install media do not come with them (for size reasons), the debug and xdebug sets can be downloaded and extracted as needed later. They provide full symbol information for all base system and X binaries and libraries and allow better error reporting and (userland) crash analysis.

    • PaX MPROTECT (W^X) memory protection enforced by default on some architectures with fine-grained memory protection and suitable ELF formats: i386, amd64, evbarm, landisk.

    • PaX ASLR (Address Space Layout Randomization) enabled by default on: i386, amd64, evbarm, landisk, sparc64.

    • Position independent executables by default for userland on: i386, amd64, arm, m68k, mips, sh3, sparc64.

    • A new socket layer can(4) has been added for communication of devices on a CAN bus.

    • A special pseudo interface ipsecif(4) for route-based VPNs has been added.

    • Parts of the network stack have been made MP-safe. The kernel option NET_MPSAFE is required to enable this.

    • Hardening of the network stack in general.

    • Various WAPBL (the NetBSD file system “log” option) stability and performance improvements.

    • Specific to i386 and amd64 CPUs:

    • Meltdown mitigation: SVS (Separate Virtual Space), enabled by default.

    • SpectreV2 mitigation: retpoline (support in gcc), used by default for kernels. Other hardware mitigations are also available.

    • SpectreV4 mitigations available for Intel and AMD.

    • PopSS workaround: user access to debug registers is turned off by default.

    • Lazy FPU saving disabled on vulnerable Intel CPUs (“eagerfpu”).

    • SMAP support.

    • Improvement and hardening of the memory layout: W^X, fewer writable pages, better consistency, better performance.

    • (U)EFI bootloader.

    • Many evbarm kernels now use FDT (flat device tree) information (loadable at boot time from an external file) for device configuration, the number of kernels has decreased but the number of boards has vastly increased.

    • Lots of updates to 3rd party software included:

    • GCC 5.5 with support for Address Sanitizer and Undefined Behavior Sanitizer

    • GDB 7.12

    • GNU binutils 2.27

    • Clang/LLVM 3.8.1

    • OpenSSH 7.6

    • OpenSSL 1.0.2k

    • mdocml 1.14.1

    • acpica 20170303

    • ntp 4.2.8p11-o

    • dhcpcd 7.0.6

    • Lua 5.3.4

    ###Running FreeBSD on the ARM64 VPS from Scaleway

    I’ve been thinking about this 6 since 2017, but only yesterday signed up for an account and played around with the ARM64 offering.
    Turns out it’s pretty great! KVM boots into UEFI, there’s a local VirtIO disk attached, no NBD junk required. So we can definitely run FreeBSD.
    I managed to “depenguinate” a running instance, the notes are below. Would be great if Scaleway offered an official image instead :wink:
    For some reason, unlike on x86 4, mounting additional volumes is not allowed 4 on ARM64 instances. So we’ll have to move the running Linux to a ramdisk using pivotroot and then we can do whatever to our one and only disk.
    Spin up an instance with Ubuntu Zesty and ssh in.

    • Prepare the system and change the root to a tmpfs:
    apt install gdisk mount -t tmpfs tmpfs /tmp cp -r /bin /sbin /etc /dev /root /home /lib /run /usr /var /tmp mkdir /tmp/proc /tmp/sys /tmp/oldroot mount /dev/vda /tmp/oldroot mount --make-rprivate / pivotroot /tmp /tmp/oldroot for i in dev proc sys run; do mount --move /oldroot/$i /$i; done systemctl daemon-reload systemctl restart sshd

    Now reconnect to ssh from a second terminal (note: rm the connection file if you use ControlPersist in ssh config), then exit the old session. Kill the old sshd process, restart or stop the rest of the stuff using the old disk:

    pkill -f notty sed -ibak 's/RefuseManualStart.$//g' /lib/systemd/system/dbus.service systemctl daemon-reload systemctl restart dbus systemctl daemon-reexec systemctl stop user@0 ntp cron systemd-logind systemctl restart systemd-journald systemd-udevd pkill agetty pkill rsyslogd

    Check that nothing is touching /oldroot:

    lsof | grep oldroot

    There will probably be an old dbus-daemon, kill it.
    And finally, unmount the old root and overwrite the hard disk with a memstick image:

    umount -R /oldroot wget https://download.freebsd.org/ftp/snapshots/arm64/aarch64/ISO-IMAGES/12.0/FreeBSD-12.0-CURRENT-arm64-aarch64-20180719-r336479-mini-memstick.img.xz xzcat FreeBSD-12.0-CURRENT-arm64-aarch64-20180719-r336479-mini-memstick.img.xz | dd if=/dev/stdin of=/dev/vda bs=1M

    (Look for the newest snapshot, don’t copy paste the July 19 link above if you’re reading this in the future. Actually maybe use a release instead of CURRENT…)
    Now, fix the GPT: move the secondary table to the end of the disk and resize the table.
    It’s important to resize here, as FreeBSD does not do that and silently creates partitions that won’t persist across reboots

    gdisk /dev/vda x e s 4 w y

    And reboot. (You might actually want to hard reboot here: for some reason on the first reboot from Linux, pressing the any-key to enter the prompt in the loader hangs the console for me.)

    I didn’t have to go into the ESC menu and choose the local disk in the boot manager, it seems to boot from disk automatically.

    Now we’re in the FreeBSD EFI loader.
    For some reason, the (recently fixed? 2) serial autodetection from EFI is not working correctly. Or something.
    So you don’t get console output by default.
    To fix, you have to run these commands in the boot loader command prompt:

    set console=comconsole,efi boot

    Ignore the warning about comconsole not being a valid console.
    Since there’s at least one (efi) that the loader thinks is valid, it sets the whole variable.)

    (UPD: shouldn’t be necessary in the next snapshot)

    Now it’s a regular installation process!
    When asked about partitioning, choose Shell, and manually add a partition and set up a root filesystem:

    gpart add -t freebsd-zfs -a 4k -l zroot vtbd0 zpool create -R /mnt -O mountpoint=none -O atime=off zroot /dev/gpt/zroot zfs create -o canmount=off -o mountpoint=none zroot/ROOT zfs create -o mountpoint=/ zroot/ROOT/default zfs create -o mountpoint=/usr zroot/ROOT/default/usr zfs create -o mountpoint=/var zroot/ROOT/default/var zfs create -o mountpoint=/var/log zroot/ROOT/default/var/log zfs create -o mountpoint=/usr/home zroot/home zpool set bootfs=zroot/ROOT/default zroot exit

    (In this example, I set up ZFS with a beadm-compatible layout which allows me to use Boot Environments.)

    In the post-install chroot shell, fix some configs like so:

    echo 'zfsload="YES"' >> /boot/loader.conf echo 'console="comconsole,efi"' >> /boot/loader.conf echo 'vfs.zfs.arcmax="512M"' >> /boot/loader.conf sysrc zfsenable=YES exit

    (Yeah, for some reason, the loader does not load zfs.ko’s dependency opensolaris.ko automatically here. idk what even. It does on my desktop and laptop.)

    Now you can reboot into the installed system!!

    Here’s how you can set up IPv6 (and root’s ssh key) auto configuration on boot:

    Pkg bootstrap pkg install curl curl https://raw.githubusercontent.com/scaleway/image-tools/master/bases/overlay-common/usr/local/bin/scw-metadata > /usr/local/bin/scw-metadata chmod +x /usr/local/bin/scw-metadata echo '#!/bin/sh' > /etc/rc.local echo 'PATH=/usr/local/bin:$PATH' >> /etc/rc.local echo 'eval $(scw-metadata)' >> /etc/rc.local echo 'echo $SSHPUBLICKEYS0KEY > /root/.ssh/authorizedkeys' >> /etc/rc.local echo 'chmod 0400 /root/.ssh/authorizedkeys' >> /etc/rc.local echo 'ifconfig vtnet0 inet6 $IPV6ADDRESS/$IPV6NETMASK' >> /etc/rc.local echo 'route -6 add default $IPV6GATEWAY' >> /etc/rc.local mkdir /run mkdir /root/.ssh sh /etc/rc.local

    And to fix incoming TCP connections, configure the DHCP client to change the broadcast address:

    echo 'interface "vtnet0" { supersede broadcast-address 255.255.255.255; }' >> /etc/dhclient.conf
    killall dhclient
    dhclient vtnet0

    • Other random notes:
    • keep in mind that -CURRENT snapshots come with a debugging kernel by default, which limits syscall performance by a lot, you might want to build your own 2 with config GENERIC-NODEBUG
    • also disable heavy malloc debugging features by running ln -s ‘abort:false,junk:false’ /etc/malloc.conf (yes that’s storing config in a symlink)
    • you can reuse the installer’s partition for swap

    * Digital Ocean **
    http://do.co/bsdnow

    ###Easy encrypted backups on OpenBSD with base tools

    Today’s topic is “Encrypted backups” using only OpenBSD base tools. I am planning to write a bigger article later about backups but it’s a wide topic with a lot of software to cover and a lot of explanations about the differents uses cases, needs, issues an solutions. Here I will stick on explaining how to make reliable backups for an OpenBSD system (my laptop).
    What we need is the dump command (see man 8 dump for its man page). It’s an utility to make a backup for a filesystem, it can only make a backup of one filesystem at a time. On my laptop I only backup /home partition so this solution is suitable for me while still being easy.
    Dump can do incremental backups, it means that it will only save what changed since the last backup of lower level. If you do not understand this, please refer to the dump man page.
    What is very interesting with dump is that it honors nodump flag which is an extended attribute of a FFS filesystem. One can use the command chflags nodump /home/solene/Downloads to tells dump not do save that folder (under some circumstances). By default, dump will not save thoses files, EXCEPT for a level 0 backup.

    • Important features of this backup solution:
    • save files with attributes, permissions and flags
    • can recreate a partition from a dump, restore files interactively, from a list or from its inode number (useful when you have files in lost+found)
    • one dump = one file

    My process is to make a huge dump of level 0 and keep it on a remote server, then, once a week I make a level 1 backup which will contain everything changed since the last dump of level 0, and everyday I do a level 2 backup of my files. The level 2 will contain latest files and the files changing a lot, which are often the most interesting. The level 1 backup is important because it will offload a lot of changes for the level 2.
    Let me explain: let says my full backup is 60 GB, full of pictures, sources files, GUI applications data files etc… A level 1 backup will contain every new picture, new projects, new GUI files etc… since the full backup, which will produce bigger and bigger dump over time, usually it is only 100 MB to 1GB. As I don’t add new pictures everyday or use new software everyday, the level 2 will take care of most littles changes to my data, like source code edited, little works on files etc… The level 2 backup is really small, I try to keep it under 50 MB so I can easily send it on my remote server everyday.
    One could you more dump level, up to level 9, but keep in mind that those are incremental. In my case, if I need to restore all my partition, I will need to use level 0, 1 and 2 to get up to latest backup state. If you want to restore a file deleted a few days ago, you need to remember in which level its latest version is.
    History note: dump was designed to be used with magnetic tapes.

    • See the article for the remainder of the article

    ##News Roundup
    Status of DFly server storage upgrades (Matt Dillon)

    Last month we did some storage upgrades, particularly of internet-facing machines for package and OS distribution. Yesterday we did a number of additional upgrades, described below. All using funds generously donated by everyone!

    The main repository server received a 2TB SSD to replace the HDDs it was using before. This will improve access to a number of things maintained by this server, including the mail archives, and gives the main repo server more breathing room for repository expansion. Space was at a premium before. Now there’s plenty.

    Monster, the quad socket opteron which we currently use as the database builder and repository that we export to our public grok service (grok.dragonflybsd.org) received a 512G SSD to add swap space for swapcache, to help cache the grok meta-data. It now has 600GB of swapcache configured. Over the next few weeks we will also be changing the grok updates to ping-pong between the two 4TB data drives it received in the last upgrade so we can do concurrent updates and web accesses without them tripping over each other performance-wise.

    The main developer box, Leaf, received a 2TB SSD and we are currently in the midst of migrating all the developer accounts in /home and /build from its old HDDs to its new SSD. This machine serves developer repos, developer web stuff, our home page and wiki, etc, so those will become snappier as well.

    Hard drives are becoming real dinosaurs. We still have a few left from the old days but in terms of active use the only HDDs we feel we really need to keep now are the ones we use for backups and grok data, owing to the amount of storage needed for those functions.

    Five years ago when we received the blade server that now sits in the colo, we had a small 256G SSD for root on every blade, and everything else used HDDs. To make things operate smoothly, most of that 256G root SSD was assigned to swapcache (200G of it, in fact, in most cases). Even just 2 years ago replacing all those HDDs with SSDs, even just the ones being used to actively serve data and support developers, would have been cost prohibitive. But today it isn’t and the only HDDs we really need anywhere are for backups or certain very large bits of bulk data (aka the grok source repository and index). The way things are going, even the backup drives will probably become SSDs over the next two years.

    ###iX ad spot
    OSCON 2018 Recap

    ###zpool checkpoints

    In March, to FreeBSD landed a very interesting feature called ‘zpool checkpoints’. Before we jump straight into the topic, let’s take a step back and look at another ZFS feature called ‘snapshot’. Snapshot allows us to create an image of our single file systems. This gives us the option to modify data on the dataset without the fear of losing some data.

    A very good example of how to use ZFS snapshot is during an upgrade of database schema. Let us consider a situation where we have a few scripts which change our schema. Sometimes we are unable to upgrade in one transaction (for example, when we attempt to alter a table and then update it in single transaction). If our database is on dataset, we can just snapshot it, and if something goes wrong, simply rollback the file system to its previous state.

    The problem with snapshot is that it works only on a single dataset. If we added some dataset, we wouldn’t then be able to create the snapshot which would rollback that operation. The same with changing the attributes of a dataset. If we change the compression on the dataset, we cannot rollback it. We would need to change that manually.

    Another interesting problem involves upgrading the whole operating system when we upgrade system with a new ZFS version. What if we start upgrading our dataset and our kernel begins to crash? (If you use FreeBSD, I doubt you will ever have had that experience but still…). If we rollback to the old kernel, there is a chance the dataset will stop working because the new kernel doesn’t know how to use the new features.

    Zpool checkpoints is the solution to all those problems. Instead of taking a single snapshot of the dataset, we can now take a snapshot of the whole pool. That means we will not only rollback the data but also all the metadata. If we rewind to the checkpoint, all our ZFS properties will be rolled back; the upgrade will be rolledback, and even the creation/deletion of the dataset, and the snapshot, will be rolledback.

    • Zpool Checkpoint has introduced a few simple functions:
    • For a creating checkpoint:

    zpool checkpoint <pool>

    • Rollbacks state to checkpoint and remove the checkpoint:

    zpool import -- rewind-to-checkpoint <pool>

    • Mount the pool read only - this does not rollback the data:

    zpool import --read-only=on --rewind-to-checkpoint

    • Remove the checkpoint

    zpool checkpoint --discard <pool> or zpool checkpoint -d <pool>

    • With this powerful feature we need to remember some safety rules:
    • Scrub will work only on data that isn’t in checkpool.
    • You can’t remove vdev if you have a checkpoint.
    • You can’t split mirror.
    • Reguid will not work either.
    • Create a checkpoint when one of the disks is removed…

    For me, this feature is incredibly useful, especially when upgrading an operating system, or when I need to experiment with additional data sets. If you speak Polish, I have some additional information for you. During the first Polish BSD user group meeting, I had the opportunity to give a short talk about this feature. Here you find the video of that talk, and here is the slideshow.

    I would like to offer my thanks to Serapheim Dimitropoulos for developing this feature, and for being so kind in sharing with me so many of its intricacies. If you are interested in knowing more about the technical details of this feature, you should check out Serapheim’s blog, and his video about checkpoints.

    ###g2k18 Reports

    ##Beastie Bits

    Tarsnap

    ##Feedback/Questions

    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Thu, 02 Aug 2018 00:00:00 -0700
    Episode 256: Because Computers | BSD Now 2^8
    FreeBSD ULE vs. Linux CFS, OpenBSD on Tuxedo InfinityBook, how zfs diff reports filenames efficiently, why choose FreeBSD over Linux, PS4 double free exploit, OpenBSD’s wifi autojoin, and FreeBSD jails the hard way. Win Celebrate our 256th episode with us. You can win a Mogics Power Bagel (not sponsored). To enter, go find the 4 episodes we did in December of 2017. In the opening, find the 4 letters in the bookshelf behind me. They spell different words in each of the 4 episodes. Send us these words in order to feedback@bsdnow.tv with the subject “bsdnow256” until August 8th, 2018 18:00 UTC and we’ll randomly draw the winner on the live show. We’ll then contact you to ship the item. Only one item to win. All decisions are final. Better luck next time. Headlines Battle of the Schedulers: FreeBSD ULE vs. Linux CFS Introduction This paper analyzes the impact on application performance of the design and implementation choices made in two widely used open-source schedulers: ULE, the default FreeBSD scheduler, and CFS, the default Linux scheduler. We compare ULE and CFS in otherwise identical circumstances. We have ported ULE to Linux, and use it to schedule all threads that are normally scheduled by CFS. We compare the performance of a large suite of applications on the modified kernel running ULE and on the standard Linux kernel running CFS. The observed performance differences are solely the result of scheduling decisions, and do not reflect differences in other subsystems between FreeBSD and Linux. There is no overall winner. On many workloads the two schedulers perform similarly, but for some workloads there are significant and even surprising differences. ULE may cause starvation, even when executing a single application with identical threads, but this starvation may actually lead to better application performance for some workloads. The more complex load balancing mechanism of CFS reacts more quickly to workload changes, but ULE achieves better load balance in the long run. Operating system kernel schedulers are responsible for maintaining high utilization of hardware resources (CPU cores, memory, I/O devices) while providing fast response time to latency-sensitive applications. They have to react to workload changes, and handle large numbers of cores and threads with minimal overhead [12]. This paper provides a comparison between the default schedulers of two of the most widely deployed open-source operating systems: the Completely Fair Scheduler (CFS) used in Linux, and the ULE scheduler used in FreeBSD. Our goal is not to declare an overall winner. In fact, we find that for some workloads ULE is better and for others CFS is better. Instead, our goal is to illustrate how differences in the design and the implementation of the two schedulers are reflected in application performance under different workloads. ULE and CFS are both designed to schedule large numbers of threads on large multicore machines. Scalability considerations have led both schedulers to adopt per-core run-queues. On a context switch, a core accesses only its local run-queue to find the next thread to run. Periodically and at select times, e.g., when a thread wakes up, both ULE and CFS perform load balancing, i.e., they try to balance the amount of work waiting in the run-queues of different cores. ULE and CFS, however, differ greatly in their design and implementation choices. FreeBSD ULE is a simple scheduler (2,950 lines of code in FreeBSD 11.1), while Linux CFS is much more complex (17,900 lines of code in the latest LTS Linux kernel, Linux 4.9). FreeBSD run-queues are FIFO. For load balancing, FreeBSD strives to even out the number of threads per core. In Linux, a core decides which thread to run next based on prior execution time, priority, and perceived cache behavior of the threads in its runqueue. Instead of evening out the number of threads between cores, Linux strives to even out the average amount of pending work. Performance analysis We now analyze the impact of the per-core scheduling on the performance of 37 applications. We define “performance” as follows: for database workloads and NAS applications, we compare the number of operations per second, and for the other applications we compare “execution time”. The higher the “performance”, the better a scheduler performs. Figure 5 presents the performance difference between CFS and ULE on a single core, with percentages above 0 meaning that the application executes faster with ULE than CFS. Overall, the scheduler has little influence on most workloads. Indeed, most applications use threads that all perform the same work, thus both CFS and ULE endup scheduling all of the threads in a round-robin fashion. The average performance difference is 1.5%, in favor of ULE. Still, scimark is 36% slower on ULE than CFS, and apache is 40% faster on ULE than CFS. Scimark is a single-threaded Java application. It launches one compute thread, and the Java runtime executes other Java system threads in the background (for the garbage collector, I/O, etc.). When the application is executed with ULE, the compute thread can be delayed, because Java system threads are considered interactive and get priority over the computation thread. The apache workload consists of two applications: the main server (httpd) running 100 threads, and ab, a single-threaded load injector. The performance difference between ULE and CFS is explained by different choices regarding thread preemption. In ULE, full preemption is disabled, while CFS preempts the running thread when the thread that has just been woken up has a vruntime that is much smaller than the vruntime of the currently executing thread (1ms difference in practice). In CFS, ab is preempted 2 million times during the benchmark, while it never preempted with ULE. This behavior is explained as follows: ab starts by sending 100 requests to the httpd server, and then waits for the server to answer. When ab is woken up, it checks which requests have been processed and sends new requests to the server. Since ab is single-threaded, all requests sent to the server are sent sequentially. In ULE, ab is able to send as many new requests as it has received responses. In CFS, every request sent by ab wakes up a httpd thread, which preempts ab. Conclusion Scheduling threads on a multicore machine is hard. In this paper, we perform a fair comparison of the design choices of two widely used schedulers: the ULE scheduler from FreeBSD and CFS from Linux. We show that they behave differently even on simple workloads, and that no scheduler performs better than the other on all workloads. OpenBSD 6.3 on Tuxedo InfinityBook Disclaimer: I came across the Tuxedo Computers InfinityBook last year at the Open! Conference where Tuxedo had a small booth. Previously they came to my attention since they’re a member of the OSB Alliance on whose board I’m a member. Furthermore Tuxedo Computers are a sponsor of the OSBAR which I’m part of the organizational team. OpenBSD on the Tuxedo InfinityBook I’ve asked the guys over at Tuxedo Computers whether they would be interested to have some tests with *BSD done and that I could test drive one of their machines and give feedback on what works and what does not - and possibly look into it.+ Within a few weeks they shipped me a machine and last week the InfinityBook Pro 14” arrived. Awesome. Thanks already to the folks at Tuxedo Computers. The machine arrived accompanied by lot’s of swag :) The InfinityBook is a very nice machine and allows a wide range of configuration. The configuration that was shipped to me: Intel Core i7-8550U 1x 16GB RAM 2400Mhz Crucial Ballistix Sport LT 250 GB Samsung 860 EVO (M.2 SATAIII) I used a USB-stick to boot install63.fs and re-installed the machine with OpenBSD. Full dmesg. The installation went flawlessly, the needed intel firmware is being installed after installation automatically via fw_update(1). Out of the box the graphics works and once installed the machine presents the login. Video When X starts the display is turned off for some reason. You will need to hit fn+f12 (the key with the moon on it) then the display will go on. Aside from that little nit, X works just fine and presents one the expected resolution. External video is working just fine as well. Either via hdmi output or via the mini displayport connector. The buttons for adjusting brightness (fn+f8 and fn+f9) are not working. Instead one has to use wsconsctl(8) to adjust the brightness. Networking The infinityBook has built-in ethernet, driven by re(4) And for the wireless interface the iwm(4) driver is being used. Both work as expected. ACPI Neither suspend nor hibernate work. Reporting of battery status is bogus as well. Some of the keyboard function keys work: LCD on/off works (fn+f2) Keyboard backlight dimming works (fn+f4) Volume (fn+f5 / fn+f6) works Sound The azalia chipset is being used for audio processing. Works as expected, volume can be controlled via buttons (fn+f5, fn+f6) or via mixerctl. Touchpad Can be controlled via wsconsctl(8). So far I must say, that the InfinityBook makes a nice machine - and I’m enjoying working with it. iXsystems iXsystems - Its all NAS How ZFS makes things like ‘zfs diff’ report filenames efficiently As a copy on write (file)system, ZFS can use the transaction group (txg) numbers that are embedded in ZFS block pointers to efficiently find the differences between two txgs; this is used in, for example, ZFS bookmarks. However, as I noted at the end of my entry on block pointers, this doesn’t give us a filesystem level difference; instead, it essentially gives us a list of inodes (okay, dnodes) that changed. In theory, turning an inode or dnode number into the path to a file is an expensive operation; you basically have to search the entire filesystem until you find it. In practice, if you’ve ever run ‘zfs diff’, you’ve likely noticed that it runs pretty fast. Nor is this the only place that ZFS quickly turns dnode numbers into full paths, as it comes up in ‘zpool status’ reports about permanent errors. At one level, zfs diff and zpool status do this so rapidly because they ask the ZFS code in the kernel to do it for them. At another level, the question is how the kernel’s ZFS code can be so fast. The interesting and surprising answer is that ZFS cheats, in a way that makes things very fast when it works and almost always works in normal filesystems and with normal usage patterns. The cheat is that ZFS dnodes record their parent’s object number. If you’re familiar with the twists and turns of Unix filesystems, you’re now wondering how ZFS deals with hardlinks, which can cause a file to be in several directories at once and so have several parents (and then it can be removed from some of the directories). The answer is that ZFS doesn’t; a dnode only ever tracks a single parent, and ZFS accepts that this parent information can be inaccurate. I’ll quote the comment in zfsobjto_pobj: When a link is removed [the file’s] parent pointer is not changed and will be invalid. There are two cases where a link is removed but the file stays around, when it goes to the delete queue and when there are additional links. Before I get into the details, I want to say that I appreciate the brute force elegance of this cheat. The practical reality is that most Unix files today don’t have extra hardlinks, and when they do most hardlinks are done in ways that won’t break ZFS’s parent stuff. The result is that ZFS has picked an efficient implementation that works almost all of the time; in my opinion, the great benefit we get from having it around are more than worth the infrequent cases where it fails or malfunctions. Both zfs diff and having filenames show up in zpool status permanent error reports are very useful (and there may be other cases where this gets used). The current details are that any time you hardlink a file to somewhere or rename it, ZFS updates the file’s parent to point to the new directory. Often this will wind up with a correct parent even after all of the dust settles; for example, a common pattern is to write a file to an initial location, hardlink it to its final destination, and then remove the initial location version. In this case, the parent will be correct and you’ll get the right name. News Roundup What is FreeBSD? Why Should You Choose It Over Linux? Not too long ago I wondered if and in what situations FreeBSD could be faster than Linux and we received a good amount of informative feedback. So far, Linux rules the desktop space and FreeBSD rules the server space. In the meantime, though, what exactly is FreeBSD? And at what times should you choose it over a GNU/Linux installation? Let’s tackle these questions. FreeBSD is a free and open source derivative of BSD (Berkeley Software Distribution) with a focus on speed, stability, security, and consistency, among other features. It has been developed and maintained by a large community ever since its initial release many years ago on November 1, 1993. BSD is the version of UNIX® that was developed at the University of California in Berkeley. And being a free and open source version, “Free” being a prefix to BSD is a no-brainer. What’s FreeBSD Good For? FreeBSD offers a plethora of advanced features and even boasts some not available in some commercial Operating Systems. It makes an excellent Internet and Intranet server thanks to its robust network services that allow it to maximize memory and work with heavy loads to deliver and maintain good response times for thousands of simultaneous user processes. FreeBSD runs a huge number of applications with ease. At the moment, it has over 32,000 ported applications and libraries with support for desktop, server, and embedded environments. with that being said, let me also add that FreeBSD is excellent for working with advanced embedded platforms. Mail and web appliances, timer servers, routers, MIPS hardware platforms, etc. You name it! FreeBSD is available to install in several ways and there are directions to follow for any method you want to use; be it via CD-ROM, over a network using NFS or FTP, or DVD. FreeBSD is easy to contribute to and all you have to do is to locate the section of the FreeBSD code base to modify and carefully do a neat job. Potential contributors are also free to improve on its artwork and documentation, among other project aspects. FreeBSD is backed by the FreeBSD Foundation, a non-profit organization that you can contribute to financially and all direct contributions are tax deductible. FreeBSD’s license allows users to incorporate the use of proprietary software which is ideal for companies interested in generating revenues. Netflix, for example, could cite this as one of the reasons for using FreeBSD servers. Why Should You Choose It over Linux? From what I’ve gathered about both FreeBSD and Linux, FreeBSD has a better performance on servers than Linux does. Yes, its packaged applications are configured to offer better a performance than Linux and it is usually running fewer services by default, there really isn’t a way to certify which is faster because the answer is dependent on the running hardware and applications and how the system is tuned. FreeBSD is reportedly more secure than Linux because of the way the whole project is developed and maintained. Unlike with Linux, the FreeBSD project is controlled by a large community of developers around the world who fall into any of these categories; core team, contributors, and committers. FreeBSD is much easier to learn and use because there aren’t a thousand and one distros to choose from with different package managers, DEs, etc. FreeBSD is more convenient to contribute to because it is the entire OS that is preserved and not just the kernel and a repo as is the case with Linux. You can easily access all of its versions since they are sorted by release numbers. Apart from the many documentations and guides that you can find online, FreeBSD has a single official documentation wherein you can find the solution to virtually any issue you will come across. So, you’re sure to find it resourceful. FreeBSD has close to no software issues compared to Linux because it has Java, is capable of running Windows programs using Wine, and can run .NET programs using Mono. FreeBSD’s ports/packages system allows you to compile software with specific configurations, thereby avoiding conflicting dependency and version issues. Both the FreeBSD and GNU/Linux project are always receiving updates. The platform you decide to go with is largely dependent on what you want to use it for, your technical know-how, willingness to learn new stuff, and ultimately your preference. What is your take on the topic? For what reasons would you choose FreeBSD over Linux if you would? Let us know what you think about both platforms in the comments section below. PS4 5.05 BPF Double Free Kernel Exploit Writeup Introduction Welcome to the 5.0x kernel exploit write-up. A few months ago, a kernel vulnerability was discovered by qwertyoruiopz and an exploit was released for BPF which involved crafting an out-of-bounds (OOB) write via use-after-free (UAF) due to the lack of proper locking. It was a fun bug, and a very trivial exploit. Sony then removed the write functionality from BPF, so that exploit was patched. However, the core issue still remained (being the lack of locking). A very similar race condition still exists in BPF past 4.55, which we will go into detail below on. The full source of the exploit can be found here. This bug is no longer accessible however past 5.05 firmware, because the BPF driver has finally been blocked from unprivileged processes - WebKit can no longer open it. Sony also introduced a new security mitigation in 5.0x firmwares to prevent the stack pointer from pointing into user space, however we’ll go more in detail on this a bit further down. Assumptions Some assumptions are made of the reader’s knowledge for the writeup. The avid reader should have a basic understanding of how memory allocators work - more specifically, how malloc() and free() allocate and deallocate memory respectively. They should also be aware that devices can be issued commands concurrently, as in, one command could be received while another one is being processed via threading. An understanding of C, x86, and exploitation basics is also very helpful, though not necessarily required. Background This section contains some helpful information to those newer to exploitation, or are unfamiliar with device drivers, or various exploit techniques such as heap spraying and race conditions. Feel free to skip to the “A Tale of Two Free()'s” section if you’re already familiar with this material. What Are Drivers? There are a few ways that applications can directly communicate with the operating system. One of which is system calls, which there are over 600 of in the PS4 kernel, ~500 of which are FreeBSD - the rest are Sony-implemented. Another method is through something called “Device Drivers”. Drivers are typically used to bridge the gap between software and hardware devices (usb drives, keyboard/mouse, webcams, etc) - though they can also be used just for software purposes. There are a few operations that a userland application can perform on a driver (if it has sufficient permissions) to interface with it after opening it. In some instances, one can read from it, write to it, or in some cases, issue more complex commands to it via the ioctl() system call. The handlers for these commands are implemented in kernel space - this is important, because any bugs that could be exploited in an ioctl handler can be used as a privilege escalation straight to ring0 - typically the most privileged state. Drivers are often the more weaker points of an operating system for attackers, because sometimes these drivers are written by developers who don’t understand how the kernel works, or the drivers are older and thus not wise to newer attack methods. The BPF Device Driver If we take a look around inside of WebKit’s sandbox, we’ll find a /dev directory. While this may seem like the root device driver path, it’s a lie. Many of the drivers that the PS4 has are not exposed to this directory, but rather only ones that are needed for WebKit’s operation (for the most part). For some reason though, BPF (aka. the “Berkely Packet Filter”) device is not only exposed to WebKit’s sandbox - it also has the privileges to open the device as R/W. This is very odd, because on most systems this driver is root-only (and for good reason). If you want to read more into this, refer to my previous write-up with 4.55FW. What Are Packet Filters? Below is an excerpt from the 4.55 bpfwrite writeup. Since the bug is directly in the filter system, it is important to know the basics of what packet filters are. Filters are essentially sets of pseudo-instructions that are parsed by bpf_filter() (which are ran when packets are received). While the pseudo-instruction set is fairly minimal, it allows you to do things like perform basic arithmetic operations and copy values around inside it’s buffer. Breaking down the BPF VM in it’s entirety is far beyond the scope of this write-up, just know that the code produced by it is ran in kernel mode - this is why read/write access to /dev/bpf should be privileged. Race Conditions Race conditions occur when two processes/threads try to access a shared resource at the same time without mutual exclusion. The problem was ultimately solved by introducing concepts such as the “mutex” or “lock”. The idea is when one thread/process tries to access a resource, it will first acquire a lock, access it, then unlock it once it’s finished. If another thread/process tries to access it while the other has the lock, it will wait until the other thread is finished. This works fairly well - when it’s used properly. Locking is hard to get right, especially when you try to implement fine-grained locking for performance. One single instruction or line of code outside the locking window could introduce a race condition. Not all race conditions are exploitable, but some are (such as this one) - and they can give an attacker very powerful bugs to work with. Heap Spraying The process of heap spraying is fairly simple - allocate a bunch of memory and fill it with controlled data in a loop and pray your allocation doesn’t get stolen from underneath you. It’s a very useful technique when exploiting something such as a use-after-free(), as you can use it to get controlled data into your target object’s backing memory. By extension, it’s useful to do this for a double free() as well, because once we have a stale reference, we can use a heap spray to control the data. Since the object will be marked “free” - the allocator will eventually provide us with control over this memory, even though something else is still using it. That is, unless, something else has already stolen the pointer from you and corrupts it - then you’ll likely get a system crash, and that’s no fun. This is one factor that adds to the variance of exploits, and typically, the smaller the object, the more likely this is to happen. Follow the link to read more of the article DigitalOcean http://do.co/bsdnow OpenBSD gains Wi-Fi “auto-join” In a change which is bound to be welcomed widely, -current has gained “auto-join” for Wi-Fi networks. Peter Hessler (phessler@) has been working on this for quite some time and he wrote about it in his p2k18 hackathon report. He has committed the work from the g2k18 hackathon in Ljubljana: CVSROOT: /cvs Module name: src Changes by: phessler@cvs.openbsd.org 2018/07/11 14:18:09 Modified files: sbin/ifconfig : ifconfig.8 ifconfig.c sys/net80211 : ieee80211ioctl.c ieee80211ioctl.h ieee80211node.c ieee80211node.h ieee80211_var.h Log message: Introduce 'auto-join' to the wifi 802.11 stack. This allows a system to remember which ESSIDs it wants to connect to, any relevant security configuration, and switch to it when the network we are currently connected to is no longer available. Works when connecting and switching between WPA2/WPA1/WEP/clear encryptions. example hostname.if: join home wpakey password join work wpakey mekmitasdigoat join open-lounge join cafe wpakey cafe2018 join "wepnetwork" nwkey "12345" dhcp inet6 autoconf up OK stsp@ reyk@ and enthusiasm from every hackroom I've been in for the last 3 years The usage should be clear from the commit message, but basically you ‘join’ all the networks you want to auto-join as you would previously use ‘nwid’ to connect to one specific network. Then the kernel will join the network that’s actually in range and do the rest automagically for you. When you move out of range of that network you lose connectivity until you come in range of the original (where things will continue to work as you’ve been used to) or one of the other networks (where you will associate and then get a new lease). Thanks to Peter for working on this feature - something many a Wi-Fi using OpenBSD user will be able to benefit from. FreeBSD Jails the hard way There are many great options for managing FreeBSD Jails. iocage, warden and ez-jail aim to streamline the process and make it quick an easy to get going. But sometimes the tools built right into the OS are overlooked. This post goes over what is involved in creating and managing jails using only the tools built into FreeBSD. For this guide, I’m going to be putting my jails in /usr/local/jails. I’ll start with a very simple, isolated jail. Then I’ll go over how to use ZFS snapshots, and lastly nullfs mounts to share the FreeBSD base files with multiple jails. I’ll also show some examples of how to use the templating power of jail.conf to apply similar settings to all your jails. Full Jail Make a directory for the jail, or a zfs dataset if you prefer. Download the FreeBSD base files, and any other parts of FreeBSD you want. In this example I’ll include the 32 bit libraries as well. Update your FreeBSD base install. Verify your download. We’re downloading these archives over FTP after all, we should confirm that this download is valid and not tampered with. The freebsd-update IDS command verifies the installation using a PGP key which is in your base system, which was presumably installed with an ISO that you verified using the FreeBSD signed checksums. Admittedly this step is a bit of paranoia, but I think it’s prudent. Make sure you jail has the right timezone and dns servers and a hostname in rc.conf. Edit jail.conf with the details about your jail. Start and login to your jail. 11 commands and a config file, but this is the most tedious way to make a jail. With a little bit of templating it can be even easier. So I’ll start by making a template. Making a template is basically the same as steps 1, 2 and 3 above, but with a different destination folder, I’ll condense them here. Creating a template Create a template or a ZFS dataset. If you’d like to use the zfs clone method of deploying templates, you’ll need to create a zfs dataset instead of a folder. Update your template with freebsd-update. Verify your install And that’s it, now you have a fully up to date jail template. If you’ve made this template with zfs, you can easily deploy it using zfs snapshots. Deploying a template with ZFS snapshots Create a snapshot. My last freebsd-update to my template brought it to patch level 17, so I’ll call my snapshot p10. Clone the snapshot to a new jail. Configure the jail hostname. Add the jail definition to jail.conf, make sure you have the global jail settings from jail.conf listed in the fulljail example. Start the jail. The downside with the zfs approach is that each jail is now a fully independent, and if you need to update your jails, you have to update them all individually. By sharing a template using nullfs mounts you can have only one copy of the base system that only needs to be updated once. Follow the link to see the rest of the article about Thin jails using NullFS mounts Simplifying jail.conf Hopefully this has helped you understand the process of how to create and manage FreeBSD jails without tools that abstract away all the details. Those tools are often quite useful, but there is always benefit in learning to do things the hard way. And in this case, the hard way doesn’t seem to be that hard after all. Beastie Bits Meetup in Zurich #4, July edition (July 19) – Which you likely missed, but now you know to look for the August edition! The next two BSD-PL User group meetings in Warsaw have been scheduled for July 30th and Aug 9th @ 1830 CEST – Submit your topic proposals now Linux Geek Books - Humble Bundle Extend loader(8) geli support to all architectures and all disk-like devices Upgrading from a bootpool to a single encrypted pool – skip the gptzfsboot part, and manually update your EFI partition with loader.efi The pkgsrc 2018Q2 for Illumos is available with 18500+ binary packages NetBSD ARM64 Images Available with SMP for RPi3 / NanoPi / Pine64 Boards Recently released CDE 2.3.0 running on Tribblix (Illumos) An Interview With Tech & Science Fiction Author Michael W Lucas A reminder : MeetBSD CFP EuroBSDCon talk acceptances have gone out, and once the tutorials are confirmed, registration will open. That will likely have happened by time you see this episode, so go register! See you in Romania Tarsnap Feedback/Questions Wilyarti - Adblocked on FreeBSD Continued… Andrew - A Question and a Story Matthew - Thanks Brian - PCI-E Controller Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Tue, 24 Jul 2018 22:00:00 -0700
    Episode 255: What Are You Pointing At | BSD Now 255
    What ZFS blockpointers are, zero-day rewards offered, KDE on FreeBSD status, new FreeBSD core team, NetBSD WiFi refresh, poor man’s CI, and the power of Ctrl+T.

    ##Headlines
    What ZFS block pointers are and what’s in them

    I’ve mentioned ZFS block pointers in the past; for example, when I wrote about some details of ZFS DVAs, I said that DVAs are embedded in block pointers. But I’ve never really looked carefully at what is in block pointers and what that means and implies for ZFS.

    The very simple way to describe a ZFS block pointer is that it’s what ZFS uses in places where other filesystems would simply put a block number. Just like block numbers but unlike things like ZFS dnodes, a block pointer isn’t a separate on-disk entity; instead it’s an on disk data format and an in memory structure that shows up in other things. To quote from the (draft and old) ZFS on-disk specification (PDF):

    A block pointer (blkptr_t) is a 128 byte ZFS structure used to physically locate, verify, and describe blocks of data on disk.

    Block pointers are embedded in any ZFS on disk structure that points directly to other disk blocks, both for data and metadata. For instance, the dnode for a file contains block pointers that refer to either its data blocks (if it’s small enough) or indirect blocks, as I saw in this entry. However, as I discovered when I paid attention, most things in ZFS only point to dnodes indirectly, by giving their object number (either in a ZFS filesystem or in pool-wide metadata).

    So what’s in a block pointer itself? You can find the technical details for modern ZFS in spa.h, so I’m going to give a sort of summary. A regular block pointer contains:

    • various metadata and flags about what the block pointer is for and what parts of it mean, including what type of object it points to.
    • Up to three DVAs that say where to actually find the data on disk. There can be more than one DVA because you may have set the copies property to 2 or 3, or this may be metadata (which normally has two copies and may have more for sufficiently important metadata).
    • The logical size (size before compression) and ‘physical’ size (the nominal size after compression) of the disk block. The physical size can do odd things and is not necessarily the asize (allocated size) for the DVA(s).
    • The txgs that the block was born in, both logically and physically (the physical txg is apparently for dva[0]). The physical txg was added with ZFS deduplication but apparently also shows up in vdev removal.
    • The checksum of the data the block pointer describes. This checksum implicitly covers the entire logical size of the data, and as a result you must read all of the data in order to verify it. This can be an issue on raidz vdevs or if the block had to use gang blocks.

    Just like basically everything else in ZFS, block pointers don’t have an explicit checksum of their contents. Instead they’re implicitly covered by the checksum of whatever they’re embedded in; the block pointers in a dnode are covered by the overall checksum of the dnode, for example. Block pointers must include a checksum for the data they point to because such data is ‘out of line’ for the containing object.

    (The block pointers in a dnode don’t necessarily point straight to data. If there’s more than a bit of data in whatever the dnode covers, the dnode’s block pointers will instead point to some level of indirect block, which itself has some number of block pointers.)

    There is a special type of block pointer called an embedded block pointer. Embedded block pointers directly contain up to 112 bytes of data; apart from the data, they contain only the metadata fields and a logical birth txg. As with conventional block pointers, this data is implicitly covered by the checksum of the containing object.

    Since block pointers directly contain the address of things on disk (in the form of DVAs), they have to change any time that address changes, which means any time ZFS does its copy on write thing. This forces a change in whatever contains the block pointer, which in turn ripples up to another block pointer (whatever points to said containing thing), and so on until we eventually reach the Meta Object Set and the uberblock. How this works is a bit complicated, but ZFS is designed to generally make this a relatively shallow change with not many levels of things involved (as I discovered recently).

    As far as I understand things, the logical birth txg of a block pointer is the transaction group in which the block pointer was allocated. Because of ZFS’s copy on write principle, this means that nothing underneath the block pointer has been updated or changed since that txg; if something changed, it would have been written to a new place on disk, which would have forced a change in at least one DVA and thus a ripple of updates that would update the logical birth txg.

    However, this doesn’t quite mean what I used to think it meant because of ZFS’s level of indirection. If you change a file by writing data to it, you will change some of the file’s block pointers, updating their logical birth txg, and you will change the file’s dnode. However, you won’t change any block pointers and thus any logical birth txgs for the filesystem directory the file is in (or anything else up the directory tree), because the directory refers to the file through its object number, not by directly pointing to its dnode. You can still use logical birth txgs to efficiently find changes from one txg to another, but you won’t necessarily get a filesystem level view of these changes; instead, as far as I can see, you will basically get a view of what object(s) in a filesystem changed (effectively, what inode numbers changed).

    (ZFS has an interesting hack to make things like ‘zfs diff’ work far more efficiently than you would expect in light of this, but that’s going to take yet another entry to cover.)

    ###Rewards of Up to $500,000 Offered for FreeBSD, OpenBSD, NetBSD, Linux Zero-Days

    Exploit broker Zerodium is offering rewards of up to $500,000 for zero-days in UNIX-based operating systems like OpenBSD, FreeBSD, NetBSD, but also for Linux distros such as Ubuntu, CentOS, Debian, and Tails.
    The offer, first advertised via Twitter earlier this week, is available as part of the company’s latest zero-day acquisition drive. Zerodium is known for buying zero-days and selling them to government agencies and law enforcement.
    The company runs a regular zero-day acquisition program through its website, but it often holds special drives with more substantial rewards when it needs zero-days of a specific category.

    • BSD zero-day rewards will be on par with Linux payouts

    The US-based company held a previous drive with increased rewards for Linux zero-days in February, with rewards going as high as $45,000.
    In another zero-day acquisition drive announced on Twitter this week, the company said it was looking again for Linux zero-days, but also for exploits targeting BSD systems. This time around, rewards can go up to $500,000, for the right exploit.
    Zerodium told Bleeping Computer they’ll be aligning the temporary rewards for BSD systems with their usual payouts for Linux distros.
    The company’s usual payouts for Linux privilege escalation exploits can range from $10,000 to $30,000. Local privilege escalation (LPE) rewards can even reach $100,000 for “an exploit with an exceptional quality and coverage,” such as, for example, a Linux kernel exploit affecting all major distributions.
    Payouts for Linux remote code execution (RCE) exploits can bring in from $50,000 to $500,000 depending on the targeted software/service and its market share. The highest rewards are usually awarded for LPEs and RCEs affecting CentOS and Ubuntu distros.

    • Zero-day price varies based on exploitation chain

    The acquisition price of a submitted zero-day is directly tied to its requirements in terms of user interaction (no click, one click, two clicks, etc.), Zerodium said.
    Other factors include the exploit reliability, its success rate, the number of vulnerabilities chained together for the final exploit to work (more chained bugs means more chances for the exploit to break unexpectedly), and the OS configuration needed for the exploit to work (exploits are valued more if they work against default OS configs).

    • Zero-days in servers “can reach exceptional amounts”

    “Price difference between systems is mostly driven by market shares,” Zerodium founder Chaouki Bekrar told Bleeping Computer via email.
    Asked about the logic behind these acquisition drives that pay increased rewards, Bekrar told Bleeping Computer the following:
    "Our aim is to always have, at any time, two or more fully functional exploits for every major software, hardware, or operating systems, meaning that from time to time we would promote a specific software/system on our social media to acquire new codes and strengthen our existing capabilities or extend them.”
    “We may also react to customers’ requests and their operational needs,” Bekrar said.

    • It’s becoming a crowded market

    Since Zerodium drew everyone’s attention to the exploit brokerage market in 2015, the market has gotten more and more crowded, but also more sleazy, with some companies being accused of selling zero-days to government agencies in countries with oppressive or dictatorial regimes, where they are often used against political oponents, journalists, and dissidents, instead of going after real criminals.
    The latest company who broke into the zero-day brokerage market is Crowdfense, who recently launched an acquisition program with prizes of $10 million, of which it already paid $4.5 million to researchers.

    Twitter Announcement

    Digital Ocean
    http://do.co/bsdnow

    ###KDE on FreeBSD – June 2018

    The KDE-FreeBSD team (a half-dozen hardy individuals, with varying backgrounds and varying degrees of involvement depending on how employment is doing) has a status message in the #kde-freebsd channel on freenode. Right now it looks like this:

    http://FreeBSD.kde.org | Bleeding edge http://FreeBSD.kde.org/area51.php | Released: Qt 5.10.1, KDE SC 4.14.3, KF5 5.46.0, Applications 18.04.1, Plasma-5.12.5, Kdevelop-5.2.1, Digikam-5.9.0

    It’s been a while since I wrote about KDE on FreeBSD, what with Calamares and third-party software happening as well. We’re better at keeping the IRC topic up-to-date than a lot of other sources of information (e.g. the FreeBSD quarterly reports, or the f.k.o website, which I’ll just dash off and update after writing this).

    • In no particular order:
    • Qt 5.10 is here, in a FrankenEngine incarnation: we still use WebEnging from Qt 5.9 because — like I’ve said before — WebEngine is such a gigantic pain in the butt to update with all the necessary patches to get it to compile.
    • Our collection of downstream patches to Qt 5.10 is growing, slowly. None of them are upstreamable (e.g. libressl support) though.
    • KDE Frameworks releases are generally pushed to ports within a week or two of release. Actually, now that there is a bigger stack of KDE software in FreeBSD ports the updates take longer because we have to do exp-runs.
    • Similarly, Applications and Plasma releases are reasonably up-to-date. We dodged a bullet by not jumping on Plasma 5.13 right away, I see. Tobias is the person doing almost all of the drudge-work of these updates, he deserves a pint of something in Vienna this summer.
    • The freebsd.kde.org website has been slightly updated; it was terribly out-of-date.

    So we’re mostly-up-to-date, and mostly all packaged up and ready to go. Much of my day is spent in VMs packaged by other people, but it’s good to have a full KDE developer environment outside of them as well. (PS. Gotta hand it to Tomasz for the amazing application for downloading and displaying a flamingo … niche usecases FTW)

    ##News Roundup
    New FreeBSD Core Team Elected

    Active committers to the project have elected your tenth FreeBSD Core
    Team.

    • Allan Jude (allanjude)
    • Benedict Reuschling (bcr)
    • Brooks Davis (brooks)
    • Hiroki Sato (hrs)
    • Jeff Roberson (jeff)
    • John Baldwin (jhb)
    • Kris Moore (kmoore)
    • Sean Chittenden (seanc)
    • Warner Losh (imp)

    Let’s extend our gratitude to the outgoing Core Team members:

    • Baptiste Daroussin (bapt)
    • Benno Rice (benno)
    • Ed Maste (emaste)
    • George V. Neville-Neil (gnn)
    • Matthew Seaman (matthew)

    Matthew, after having served as the Core Team Secretary for the past
    four years, will be stepping down from that role.

    The Core Team would also like to thank Dag-Erling Smørgrav for running a
    flawless election.

    ###NetBSD WiFi refresh

    The NetBSD Foundation is pleased to announce a summer 2018 contract with Philip Nelson (phil%NetBSD.org@localhost) to update the IEEE 802.11 stack basing the update on the FreeBSD current code. The goals of the project are:

    • Minimizing the differences between the FreeBSD and NetBSD IEEE 802.11 stack so future updates are easier.
    • Adding support for the newer protocols 801.11/N and 802.11/AC.
    • Improving SMP support in the IEEE 802.11 stack.
    • Adding Virtual Access Point (VAP) support.
    • Updating as many NIC drivers as time permits for the updated IEEE 802.11 stack and VAP changes.

    Status reports will be posted to tech-net%NetBSD.org@localhost every other week
    while the contract is active.

    iXsystems

    ###Poor Man’s CI - Hosted CI for BSD with shell scripting and duct tape

    Poor Man’s CI (PMCI - Poor Man’s Continuous Integration) is a collection of scripts that taken together work as a simple CI solution that runs on Google Cloud. While there are many advanced hosted CI systems today, and many of them are free for open source projects, none of them seem to offer a solution for the BSD operating systems (FreeBSD, NetBSD, OpenBSD, etc.)

    The architecture of Poor Man’s CI is system agnostic. However in the implementation provided in this repository the only supported systems are FreeBSD and NetBSD. Support for additional systems is possible.

    Poor Man’s CI runs on the Google Cloud. It is possible to set it up so that the service fits within the Google Cloud “Always Free” limits. In doing so the provided CI is not only hosted, but is also free! (Disclaimer: I am not affiliated with Google and do not otherwise endorse their products.)

    • ARCHITECTURE

    A CI solution listens for “commit” (or more usually “push”) events, builds the associated repository at the appropriate place in its history and reports the results. Poor Man’s CI implements this very basic CI scenario using a simple architecture, which we present in this section.

    • Poor Man’s CI consists of the following components and their interactions:

    • Controller: Controls the overall process of accepting GitHub push events and starting builds. The Controller runs in the Cloud Functions environment and is implemented by the files in the controller source directory. It consists of the following components:

      • Listener: Listens for GitHub push events and posts them as work messages to the workq PubSub.
      • Dispatcher: Receives work messages from the workq PubSub and a free instance name from the Builder Pool. It instantiates a builder instance named name in the Compute Engine environment and passes it the link of a repository to build.
      • Collector: Receives done messages from the doneq PubSub and posts the freed instance name back to the Builder Pool.
    • PubSub Topics:

      • workq: Transports work messages that contain the link of the repository to build.
      • poolq: Implements the Builder Pool, which contains the name’s of available builder instances. To acquire a builder name, pull a message from the poolq. To release a builder name, post it back into the poolq.
      • doneq: Transports done messages (builder instance terminate and delete events). These message contain the name of freed builder instances.
    • builder: A builder is a Compute Engine instance that performs a build of a repository and shuts down when the build is complete. A builder is instantiated from a VM image and a startx (startup-exit) script.

    • Build Logs: A Storage bucket that contains the logs of builds performed by builder instances.

    • Logging Sink: A Logging Sink captures builder instance terminate and delete events and posts them into the doneq.

    • BUGS

    The Builder Pool is currently implemented as a PubSub; messages in the PubSub contain the names of available builder instances. Unfortunately a PubSub retains its messages for a maximum of 7 days. It is therefore possible that messages will be discarded and that your PMCI deployment will suddenly find itself out of builder instances. If this happens you can reseed the Builder Pool by running the commands below. However this is a serious BUG that should be fixed. For a related discussion see https://tinyurl.com/ybkycuub.

    $ ./pmci queuepost poolq builder0
    # ./pmci queuepost poolq builder1
    # ... repeat for as many builders as you want

    The Dispatcher is implemented as a Retry Background Cloud Function. It accepts work messages from the workq and attempts to pull a free name from the poolq. If that fails it returns an error, which instructs the infrastructure to retry. Because the infrastructure does not provide any retry controls, this currently happens immediately and the Dispatcher spins unproductively. This is currently mitigated by a “sleep” (setTimeout), but the Cloud Functions system still counts the Function as running and charges it accordingly. While this fits within the “Always Free” limits, it is something that should eventually be fixed (perhaps by the PubSub team). For a related discussion see https://tinyurl.com/yb2vbwfd.

    ###The Power of Ctrl-T

    Did you know that you can check what a process is doing by pressing CTRL+T?
    Has it happened to you before that you were waiting for something to be finished that can take a lot of time, but there is no easy way to check the status. Like a dd, cp, mv and many others. All you have to do is press CTRL+T where the process is running. This will output what’s happening and will not interrupt or mess with it in any way. This causes the operating system to output the SIGINFO signal.
    On FreeBSD it looks like this:

    ping pingtest.com PING pingtest.com (5.22.149.135): 56 data bytes 64 bytes from 5.22.149.135: icmpseq=0 ttl=51 time=86.232 ms 64 bytes from 5.22.149.135: icmpseq=1 ttl=51 time=85.477 ms 64 bytes from 5.22.149.135: icmpseq=2 ttl=51 time=85.493 ms 64 bytes from 5.22.149.135: icmpseq=3 ttl=51 time=85.211 ms 64 bytes from 5.22.149.135: icmpseq=4 ttl=51 time=86.002 ms load: 1.12 cmd: ping 94371 [select] 4.70r 0.00u 0.00s 0% 2500k 5/5 packets received (100.0%) 85.211 min / 85.683 avg / 86.232 max 64 bytes from 5.22.149.135: icmpseq=5 ttl=51 time=85.725 ms 64 bytes from 5.22.149.135: icmp_seq=6 ttl=51 time=85.510 ms

    As you can see it not only outputs the name of the running command but the following parameters as well:

    94371 – PID 4.70r – since when is the process running 0.00u – user time 0.00s – system time 0% – CPU usage 2500k – resident set size of the process or RSS `` > An even better example is with the following cp command:

    cp FreeBSD-11.1-RELEASE-amd64-dvd1.iso /dev/null
    load: 0.99 cmd: cp 94412 [runnable] 1.61r 0.00u 0.39s 3% 3100k
    FreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 15%
    load: 0.91 cmd: cp 94412 [runnable] 2.91r 0.00u 0.80s 6% 3104k
    FreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 32%
    load: 0.91 cmd: cp 94412 [runnable] 4.20r 0.00u 1.23s 9% 3104k
    FreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 49%
    load: 0.91 cmd: cp 94412 [runnable] 5.43r 0.00u 1.64s 11% 3104k
    FreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 64%
    load: 1.07 cmd: cp 94412 [runnable] 6.65r 0.00u 2.05s 13% 3104k
    FreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 79%
    load: 1.07 cmd: cp 94412 [runnable] 7.87r 0.00u 2.43s 15% 3104k
    FreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 95%

    > I prcessed CTRL+T six times. Without that, all the output would have been is the first line. > Another example how the process is changing states:

    wget https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/11.1/FreeBSD-11.1-RELEASE-amd64-dvd1.iso
    –2018-06-17 18:47:48– https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/11.1/FreeBSD-11.1-RELEASE-amd64-dvd1.iso
    Resolving download.freebsd.org (download.freebsd.org)… 96.47.72.72, 2610:1c1:1:606c::15:0
    Connecting to download.freebsd.org (download.freebsd.org)|96.47.72.72|:443… connected.
    HTTP request sent, awaiting response… 200 OK
    Length: 3348465664 (3.1G) [application/octet-stream]
    Saving to: ‘FreeBSD-11.1-RELEASE-amd64-dvd1.iso’

    FreeBSD-11.1-RELEASE-amd64-dvd1.iso 1%[> ] 41.04M 527KB/s eta 26m 49sload: 4.95 cmd: wget 10152 waiting 0.48u 0.72s
    FreeBSD-11.1-RELEASE-amd64-dvd1.iso 1%[> ] 49.41M 659KB/s eta 25m 29sload: 12.64 cmd: wget 10152 waiting 0.55u 0.85s
    FreeBSD-11.1-RELEASE-amd64-dvd1.iso 2%[=> ] 75.58M 6.31MB/s eta 20m 6s load: 11.71 cmd: wget 10152 running 0.73u 1.19s
    FreeBSD-11.1-RELEASE-amd64-dvd1.iso 2%[=> ] 85.63M 6.83MB/s eta 18m 58sload: 11.71 cmd: wget 10152 waiting 0.80u 1.32s
    FreeBSD-11.1-RELEASE-amd64-dvd1.iso 14%[==============> ] 460.23M 7.01MB/s eta 9m 0s 1

    > The bad news is that CTRl+T doesn’t work with Linux kernel, but you can use it on MacOS/OS-X:

    —> Fetching distfiles for gmp
    —> Attempting to fetch gmp-6.1.2.tar.bz2 from https://distfiles.macports.org/gmp
    —> Verifying checksums for gmp
    —> Extracting gmp
    —> Applying patches to gmp
    —> Configuring gmp
    load: 2.81 cmd: clang 74287 running 0.31u 0.28s

    > PS: If I recall correctly Feld showed me CTRL+T, thank you! Beastie Bits Half billion tries for a HAMMER2 bug (http://lists.dragonflybsd.org/pipermail/commits/2018-May/672263.html) OpenBSD with various Desktops OpenBSD 6.3 running twm window manager (https://youtu.be/v6XeC5wU2s4) OpenBSD 6.3 jwm and rox desktop (https://youtu.be/jlSK2oi7CBc) OpenBSD 6.3 cwm youtube video (https://youtu.be/mgqNyrP2CPs) pf: Increase default state table size (https://svnweb.freebsd.org/base?view=revision&revision=336221) *** Tarsnap Feedback/Questions Ben Sims - Full feed? (http://dpaste.com/3XVH91T#wrap) Scott - Questions and Comments (http://dpaste.com/08P34YN#wrap) Troels - Features of FreeBSD 11.2 that deserve a mention (http://dpaste.com/3DDPEC2#wrap) Fred - Show Ideas (http://dpaste.com/296ZA0P#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) iXsystems It's all NAS (https://www.ixsystems.com/blog/its-all-nas/)
    Wed, 18 Jul 2018 00:00:00 -0700
    Episode 254: Bare the OS | BSD Now 254
    Control flow integrity with HardenedBSD, fixing bufferbloat with OpenBSD’s pf, Bareos Backup Server on FreeBSD, MeetBSD CfP, crypto simplified interface, twitter gems, interesting BSD commits, and more.

    ##Headlines
    Silent Fanless FreeBSD Desktop/Server

    Today I will write about silent fanless FreeBSD desktop or server computer … or NAS … or you name it, it can have multa##Headlines
    ###Cross-DSO CFI in HardenedBSD
    Control Flow Integrity, or CFI, raises the bar for attackers aiming to hijack control flow and execute arbitrary code. The llvm compiler toolchain, included and used by default in HardenedBSD 12-CURRENT/amd64, supports forward-edge CFI. Backward-edge CFI support is gained via a tangential feature called SafeStack. Cross-DSO CFI builds upon ASLR and PaX NOEXEC for effectiveness.
    HardenedBSD supports non-Cross-DSO CFI in base for 12-CURRENT/amd64 and has it enabled for a few individual ports. The term “non-Cross-DSO CFI” means that CFI is enabled for code within an application’s codebase, but not for the shared libraries it depends on. Supporting non-Cross-DSO CFI is an important initial milestone for supporting Cross-DSO CFI, or CFI applied to both shared libraries and applications.
    This article discusses where HardenedBSD stands with regards to Cross-DSO CFI in base. We have made a lot of progress, yet we’re not even half-way there.
    Brace yourself: This article is going to be full of references to “Cross-DSO CFI.” Make a drinking game out of it. Or don’t. It’s your call. ;)

    • Using More llvm Toolchain Components

    CFI requires compiling source files with Link-Time Optimization (LTO). I remembered hearing a few years back that llvm developers were able to compile the entirety of FreeBSD’s source code with LTO. Compiling with LTO produces intermediate object files as LLVM IR bitcode instead of ELF objects.
    In March of 2017, we started compiling all applications with LTO and non-Cross-DSO CFI. This also enabled ld.lld as the default linker in base since CFI requires lld. Commit f38b51668efcd53b8146789010611a4632cafade made the switch to ld.lld as the default linker while enabling non-Cross-DSO CFI at the same time.
    Building libraries in base requires applications like ar, ranlib, nm, and objdump. In FreeBSD 12-CURRENT, ar and ranlib are known as “BSD ar” and “BSD ranlib.” In fact, ar and ranlib are the same applications. One is hardlinked to another and the application changes behavior depending on arvgv[0] ending in “ranlib”. The ar, nm, and objdump used in FreeBSD do not support LLVM IR bitcode object files.
    In preparation for Cross-DSO CFI support, commit fe4bb0104fc75c7216a6dafe2d7db0e3f5fe8257 in October 2017 saw HardenedBSD switching ar, ranlib, nm, and objdump to their respective llvm components. The llvm versions due support LLVM IR bitcode object files (surprise!) There has been some fallout in the ports tree and we’ve added LLVM_AR_UNSAFE and friends to help transition those ports that dislike llvm-ar, llvm-ranlib, llvm-nm, and llvm-objdump.
    With ld.lld, llvm-ar, llvm-ranlib, llvm-nm, and llvm-objdump the default, HardenedBSD has effectively switched to a full llvm compiler toolchain in 12-CURRENT/amd64.

    • Building Libraries With LTO

    The primary 12-CURRENT development branch in HardenedBSD (hardened/current/master) only builds applications with LTO as mentioned in the secion above. My first attempt at building all static and shared libraries failed due to issues within llvm itself.
    I reported these issues to FreeBSD. Ed Maste (emaste@), Dimitry Andric (dim@), and llvm’s Rafael Espindola expertly helped address these issues. Various commits within the llvm project by Rafael fully and quickly resolved the issues brought up privately in emails.
    With llvm fixed, I could now build nearly every library in base with LTO. I noticed, however, that if I kept non-Cross-DSO CFI and SafeStack enabled, all applications would segfault. Even simplistic applications like /bin/ls.
    Disabling both non-Cross-DSO CFI and SafeStack, but keeping LTO produced a fully functioning world! I have spent the last few months figuring out why enabling either non-Cross-DSO CFI or SafeStack caused issues. This brings us to today.

    • The Sanitizers in FreeBSD

    FreeBSD brought in all the files required for SafeStack and CFI. When compiling with SafeStack, llvm statically links a full sanitization framework into the application. FreeBSD includes a full copy of the sanitization framework in SafeStack, including the common C++ sanization namespaces. Thus, libclang_rt.safestack included code meant to be shared among all the sanitizers, not just SafeStack.
    I had naively taken a brute-force approach to setting up the libclang_rt.cfi static library. I copied the Makefile from libclang_rt.safestack and used that as a template for libclang_rt.cfi. This approach was incorrect due to breaking the One Definition Rule (ODR). Essentially, I ended up including a duplicate copy of the C++ classes and sanitizer runtime if both CFI and SafeStack were used.
    In my Cross-DSO CFI development VM, I now have SafeStack disabled across-the-board and am only compiling in CFI. As of 26 May 2018, an LTO-ified world (libs + apps) works in my limited testing. /bin/ls does not crash anymore! The second major milestone for Cross-DSO CFI has now been reached.

    • Known Issues And Limitations

    There are a few known issues and regressions. Note that this list of known issues essentially also constitutes a “work-in-progress” and every known issue will be fixed prior to the official launch of Cross-DSO CFI.
    It seems llvm does not like statically compiling applications with LTO that have a mixture of C and C++ code. /sbin/devd is one of these applications. As such, when Cross-DSO CFI is enabled, devd is compiled as a Position-Independent Executable (PIE). Doing this breaks UFS systems where /usr is on a separate partition. We are currently looking into solving this issue to allow devd to be statically compiled again.
    NO_SHARED is now unset in the tools build stage (aka, bootstrap-tools, cross-tools). This is related to the static compilation issue above. Unsetting NO_SHARED for to tools build stage is only a band-aid until we can resolve static compliation with LTO.
    One goal of our Cross-DSO CFI integration work is to be able to support the cfi-icall scheme when dlopen(3) and dlsym(3)/dlfunc(3) is used. This means the runtime linker (RTLD), must be enhanced to know and care about the CFI runtime. This enhancement is not currently implemented, but is planned.
    When Cross-DSO CFI is enabled, SafeStack is disabled. This is because compiling with Cross-DSO CFI brings in a second copy of the sanitizer runtime, violating the One Definition Rule (ODR). Resolving this issue should be straightforward: Unify the sanitizer runtime into a single common library that both Cross-DSO CFI and SafeStack can link against. When the installed world has Cross-DSO CFI enabled, performing a buildworld with Cross-DSO CFI disabled fails. This is somewhat related to the static compilation issue described above.

    • Current Status

    I’ve managed to get a Cross-DSO CFI world booting on bare metal (my development laptop) and in a VM. Some applications failed to work. Curiously, Firefox still worked (which also means xorg works).
    I’m now working through the known issues list, researching and learning.

    • Future Work

    Fixing pretty much everything in the “Known Issues And Limitations” section. ;P
    I need to create a static library that includes only a single copy of the common sanitizer framework code. Applications compiled with CFI or SafeStack will then only have a single copy of the framework.
    Next I will need to integrate support in the RTLD for Cross-DSO CFI. Applications with the cfi-icall scheme enabled that call functions resolved through dlsym(3) currently crash due to the lack of RTLD support. I need to make a design decision as to whether to only support adding cfi-icall whitelist entries only with dlfunc(3) or to also whitelist cfi-icall entries with the more widely used dlsym(3).
    There’s likely more items in the “TODO” bucket that I am not currently aware of. I’m treading in uncharted territory. I have no firm ETA for any bit of this work. We may gain Cross-DSO CFI support in 2018, but it’s looking like it will be later in either 2019 or 2020.

    • Conclusion

    I have been working on Cross-DSO CFI support in HardenedBSD for a little over a year now. A lot of progress is being made, yet there’s still some major hurdles to overcome. This work has already helped improve llvm and I hope more commits upstream to both FreeBSD and llvm will happen.
    We’re getting closer to being able to send out a preliminary Call For Testing (CFT). At the very least, I would like to solve the static linking issues prior to publishing the CFT. Expect it to be published before the end of 2018.
    I would like to thank Ed Maste, Dimitry Andric, and Rafael Espindola for their help, guidance, and support.

    iXsystems
    FreeNAS 11.2-BETAs are starting to appear

    ###Bareos Backup Server on FreeBSD

    Ever heard about Bareos? Probably heard about Bacula. Read what is the difference here – Why Bareos forked from Bacula?
    Bareos (Backup Archiving Recovery Open Sourced) is a network based open source backup solution. It is 100% open source fork of the backup project from bacula.org site. The fork is in development since late 2010 and it has a lot of new features. The source is published on github and licensed under AGPLv3 license. Bareos supports ‘Always Incremental backup which is interesting especially for users with big data. The time and network capacity consuming full backups only have to be taken once. Bareos comes with WebUI for administration tasks and restore file browser. Bareos can backup data to disk and to tape drives as well as tape libraries. It supports compression and encryption both hardware-based (like on LTO tape drives) and software-based. You can also get professional services and support from Bareos as well as Bareos subscription service that provides you access to special quality assured installation packages.

    I started my sysadmin job with backup system as one of the new responsibilities, so it will be like going back to the roots. As I look on the ‘backup’ market it is more and more popular – especially in cloud oriented environments – to implement various levels of protection like GOLD, SILVER and BRONZE for example. They of course have different retention times, number of backups kept, different RTO and RPO. Below is a example implementation of BRONZE level backups in Bareos. I used 3 groups of A, B and C with FULL backup starting on DAY 0 (A group), DAY 1 (B group) and DAY 2 (C group).
    This way you still have FULL backups quite often and with 3 groups you can balance the network load. I for the days that we will not be doing FULL backups we will be doing DIFFERENTIAL backups. People often confuse them with INCREMENTAL backups. The difference is that DIFFERENTIAL backups are always against FULL backup, so its always ‘one level of combining’. INCREMENTAL ones are done against last done backup TYPE, so its possible to have 100+ levels of combining against 99 earlier INCREMENTAL backups and the 1 FULL backup. That is why I prefer DIFFERENTIAL ones here, faster recovery. That is all backups is about generally, recovery, some people/companies tend to forget that.
    The implementation of BRONZE in these three groups is not perfect, but ‘does the job’. I also made ‘simulation’ how these group will overlap at the end/beginning of the month, here is the result.
    Not bad for my taste.

    Today I will show you how to install and configure Bareos Server based on FreeBSD operating system. It will be the most simplified setup with all services on single machine:

    • bareos-dir
    • bareos-sd
    • bareos-webui
    • bareos-fd

    I also assume that in order to provide storage space for the backup data itself You would mount resources from external NFS shares.

    To get in touch with Bareos terminology and technology check their great Manual in HTML or PDF version depending which format You prefer for reading documentation. Also their FAQ provides a lot of needed answers.

    Also this diagram may be useful for You to get some grip into the Bareos world.

    • System

    As every system needs to have its name we will use latin word closest to backup here – replica – for our FreeBSD system hostname. The install would be generally the same as in the FreeBSD Desktop – Part 2 – Install article. Here is our installed FreeBSD system with login prompt.

    Thu, 12 Jul 2018 08:00:00 -0700
    Episode 253: Silence of the Fans | BSD Now 253
    Fanless server setup with FreeBSD, NetBSD on pinebooks, another BSDCan trip report, transparent network audio, MirBSD's Korn Shell on Plan9, static site generators on OpenBSD, and more.

    ##Headlines
    Silent Fanless FreeBSD Desktop/Server

    Today I will write about silent fanless FreeBSD desktop or server computer … or NAS … or you name it, it can have multiple purposes. It also very low power solution, which also means that it will not overheat. Silent means no fans at all, even for the PSU. The format of the system should also be brought to minimum, so Mini-ITX seems best solution here.

    I have chosen Intel based solutions as they are very low power (6-10W), if you prefer AMD (as I often do) the closest solution in comparable price and power is Biostar A68N-2100 motherboard with AMD E1-2100 CPU and 9W power. Of course AMD has even more low power SoC solutions but finding the Mini-ITX motherboard with decent price is not an easy task. For comparison Intel has lots of such solutions below 6W whose can be nicely filtered on the ark.intel.com page. Pity that AMD does not provide such filtration for their products. I also chosen AES instructions as storage encryption (GELI on FreeBSD) today seems as obvious as HTTPS for the web pages.

    This motherboard uses Intel J3355 SoC which uses 10W and has AES instructions. It has two cores at your disposal but it also supports VT-x and EPT extensions so you can even run Bhyve on it.

    • Components

    Now, an example system would look like that one below, here are the components with their prices.

    • $49 CPU/Motherboard ASRock J3355B-ITX Mini-ITX
    • $14 RAM Crucial 4 GB DDR3L 1.35V (low power)
    • $17 PSU 12V 160W Pico (internal)
    • $11 PSU 12V 96W FSP (external)
    • $5 USB 2.0 Drive 16 GB ADATA
    • $4 USB Wireless 802.11n
    • $100 TOTAL

    The PSU 12V 160W Pico (internal) and PSU 12V 96W FSP can be purchased on aliexpress.com or ebay.com for example, at least I got them there. Here is the 12V 160W Pico (internal) PSU and its optional additional cables to power the optional HDDs. If course its one SATA power and one MOLEX power so additional MOLEX-SATA power adapter for about 1$ would be needed. Here is the 12V 96W FSP (external) PSU without the power cord.

    This gives as total silent fanless system price of about $120. Its about ONE TENTH OF THE COST of the cheapest FreeNAS hardware solution available – the FreeNAS Mini (Diskless) costs $1156 also without disks.

    You can put plain FreeBSD on top of it or Solaris/Illumos distribution OmniOSce which is server oriented. You can use prebuilt NAS solution based on FreeBSD like FreeNAS, NAS4Free, ZFSguru or even Solaris/Illumos based storage with napp-it appliance.

    ###An annotated look at a NetBSD Pinebook’s startup

    • Pinebook is an affordable 64-bit ARM notebook. Today we’re going to take a look at the kernel output at startup and talk about what hardware support is available on NetBSD.
    • Photo
    • Pinebook comes with 2GB RAM standard. A small amount of this is reserved by the kernel and framebuffer.
    • NetBSD uses flattened device-tree (FDT) to enumerate devices on all Allwinner based SoCs. On a running system, you can inspect the device tree using the ofctl(8) utility:
    • Pinebook’s Allwinner A64 processor is based on the ARM Cortex-A53. It is designed to run at frequencies up to 1.2GHz.
    • The A64 is a quad core design. NetBSD’s aarch64 pmap does not yet support SMP, so three cores are disabled for now.
    • The interrupt controller is a standard ARM GIC-400 design.
    • Clock drivers for managing PLLs, module clock dividers, clock gating, software resets, etc. Information about the clock tree is exported in the hw.clk sysctl namespace (root access required to read these values).
    # sysctl hw.clk.sun50ia64ccu0.mmc2 hw.clk.sun50ia64ccu0.mmc2.rate = 200000000 hw.clk.sun50ia64ccu0.mmc2.parent = pllperiph02x hw.clk.sun50ia64ccu0.mmc2.parent_domain = sun50ia64ccu0

    Digital Ocean
    http://do.co/bsdnow

    ###BSDCan 2018 Trip Report: Mark Johnston

    BSDCan is a highlight of my summers: the ability to have face-to-face conversations with fellow developers and contributors is invaluable and always helps refresh my enthusiasm for FreeBSD. While in a perfect world we would all be able to communicate effectively over the Internet, it’s often noted that locking a group of developers together in a room can be a very efficient way to make progress on projects that otherwise get strung out over time, and to me this is one of the principal functions of BSD conferences. In my case I was able to fix some kgdb bugs that had been hindering me for months; get some opinions on the design of a feature I’ve been working on for FreeBSD 12.0; hear about some ongoing usage of code that I’ve worked on; and do some pair-debugging of an issue that has been affecting another developer.
    As is tradition, on Tuesday night I dropped off my things at the university residence where I was staying, and headed straight to the Royal Oak. This year it didn’t seem quite as packed with BSD developers, but I did meet several long-time colleagues and get a chance to catch up. In particular, I chatted with Justin Hibbits and got to hear about the bring-up of FreeBSD on POWER9, a new CPU family released by IBM. Justin was able to acquire a workstation based upon this CPU, which is a great motivator for getting FreeBSD into shape on that platform. POWER9 also has some promise in the server market, so it’s important for FreeBSD to be a viable OS choice there.
    Wednesday morning saw the beginning of the two-day FreeBSD developer summit, which precedes the conference proper. Gordon Tetlow led the summit and did an excellent job organizing things and keeping to the schedule. The first presentation was by Deb Goodkin of the FreeBSD Foundation, who gave an overview of the Foundation’s role and activities. After Deb’s presentation, present members of the FreeBSD core team discussed the work they had done over the past two years, as well as open tasks that would be handed over to the new core team upon completion of the ongoing election. Finally, Marius Strobl rounded off the day’s presentations by discussing the state and responsibilities of FreeBSD’s release engineering team.
    One side discussion of interest to me was around the notion of tightening integration with our Bugzilla instance; at moment we do not have any good means to mark a given bug as blocking a release, making it easy for bugs to slip into releases and thus lowering our overall quality. With FreeBSD 12.0 upon us, I plan to help with the triage and fixes for known regressions before the release process begins.
    After a break, the rest of the morning was devoted to plans for features in upcoming FreeBSD releases. This is one of my favorite discussion topics and typically takes the form of have/need/want, where developers collectively list features that they’ve developed and intend to upstream (have), features that they are missing (need), and nice-to-have features (want). This year, instead of the usual format, we listed features that are intended to ship in FreeBSD 12.0. The compiled list ended up being quite ambitious given how close we are to the beginning of the release cycle, but many individual developers (including myself) have signed up to deliver work. I’m hopeful that most, if not all of it, will make it into the release.
    After lunch, I attended a discussion led by Matt Ahrens and Alexander Motin on OpenZFS. Of particular interest to me were some observations made regarding the relative quantity and quality of contributions made by different “camps” of OpenZFS users (illumos, FreeBSD and ZoL), and their respective track records of upstreaming enhancements to the OpenZFS project. In part due to the high pace of changes in ZoL, the definition of “upstream” for ZFS has become murky, and of late ZFS changes have been ported directly from ZoL. Alexander discussed some known problems with ZFS on FreeBSD that have been discovered through performance testing. While I’m not familiar with ZFS internals, Alexander noted that ZFS’ write path has poor SMP scalability on FreeBSD owing to some limitations in a certain kernel API called taskqueue(9). I would like to explore this problem further and perhaps integrate a relatively new alternative interface which should perform better.
    Friday and Saturday were, of course, taken up by BSDCan talks. Friday’s keynote was by Benno Rice, who provided some history of UNIX boot systems as a precursor to some discussion of systemd and the difficulties presented by a user and developer community that actively resist change. The rest of the morning was consumed by talks and passed by quickly. First was Colin Percival’s detailed examination of where the FreeBSD kernel spends time during boot, together with an overview of some infrastructure he added to track boot times. He also provided a list of improvements that have been made since he started taking measurements, and some areas we can further improve. Colin’s existing work in this area has already brought about substantial reductions in boot time; amusingly, one of the remaining large delays comes from the keyboard driver, which contains a workaround for old PS/2 keyboards. While there seems to be general agreement that the workaround is probably no longer needed on most systems, the lingering uncertainty around this prevents us from removing the workaround. This is, sadly, a fairly typical example of an OS maintenance burden, and underscores the need to carefully document hardware bug workarounds. After this talk, I got to see some rather novel demonstrations of system tracing using dwatch, a new utility by Devin Teske, which aims to provide a user-friendly interface to DTrace. After lunch, I attended talks on netdump, a protocol for transmitting kernel dumps over a network after the system has panicked, and on a VPC implementation for FreeBSD. After the talks ended, I headed yet again to the hacker lounge and had some fruitful discussions on early microcode loading (one of my features for FreeBSD 12.0). These led me to reconsider some aspects of my approach and saved me a lot of time. Finally, I continued my debugging session from Wednesday with help from a couple of other developers.
    Saturday’s talks included a very thorough account by Li-Wen Hsu of his work in organizing a BSD conference in Taipei last year. As one of the attendees, I had felt that the conference had gone quite smoothly and was taken aback by the number of details and pitfalls that Li-Wen enumerated during his talk. This was followed by an excellent talk by Baptiste Daroussin on the difficulties one encounters when deploying FreeBSD in new environments. Baptiste offered criticisms of a number of aspects of FreeBSD, some of which hit close to home as they involved portions of the system that I’ve worked on.
    At the conclusion of the talks, we all gathered in the main lecture hall, where Dan led a traditional and quite lively auction for charity. I managed to snag a Pine64 board and will be getting FreeBSD installed on it the first chance I get. At the end of the auction, we all headed to ByWard for dinner, concluding yet another BSDCan.

    • Thanks to Mark for sharing his experiences at this years BSDCan

    ##News Roundup
    Transparent network audio with mpd & sndiod

    Landry Breuil (landry@ when wearing his developer hat) wrote in…

    I've been a huge fan of MPD over the years to centralize my audio collection, and i've been using it with the http output to stream the music as a radio on the computer i'm currently using… audio_output { type "sndio" name "Local speakers" mixer_type "software" } audio_output { type "httpd" name "HTTP stream" mixer_type "software" encoder "vorbis" port "8000" format "44100:16:2" } this setup worked for years, allows me to stream my home radio to $work by tunnelling the port 8000 over ssh via LocalForward, but that still has some issues: a distinct timing gap between the 'local output' (ie the speakers connected to the machine where MPD is running) and the 'http output' caused by the time it takes to reencode the stream, which is ugly when you walk through the house and have a 15s delay sometimes mplayer as a client doesn't detect the pauses in the stream and needs to be restarted i need to configure/start a client on each computer and point it at the sound server url (can do via gmpc shoutcast client plugin…) it's not that elegant to reencode the stream, and it wastes cpu cycles So the current scheme is: mpd -> http output -> network -> mplayer -> sndiod on remote machine | -> sndio output -> sndiod on soundserver Fiddling a little bit with mpd outputs and reading the sndio output driver, i remembered sndiod has native network support… and the mpd sndio output allows you to specify a device (it uses SIO_DEVANY by default). So in the end, it's super easy to: enable network support in sndio on the remote machine i want the audio to play by adding -L<local ip> to sndiod_flags (i have two audio devices, with an input coming from the webcam): sndiod_flags="-L10.246.200.10 -f rsnd/0 -f rsnd/1" open pf on port 11025 from the sound server ip: pass in proto tcp from 10.246.200.1 to any port 11025 configure a new output in mpd: audio_output { type "sndio" name "sndio on renton" device "snd@10.246.200.10/0" mixer_type "software" } and enable the new output in mpd: $mpc enable 2 Output 1 (Local speakers) is disabled Output 2 (sndio on renton) is enabled Output 3 (HTTP stream) is disabled Results in a big win: no gap anymore with the local speakers, no reencoding, no need to configure a client to play the stream, and i can still probably reproduce the same scheme over ssh from $work using a RemoteForward. mpd -> sndio output 2 -> network -> sndiod on remote machine | -> sndio output 1 -> sndiod on soundserver Thanks ratchov@ for sndiod :)

    ###MirBSD’s Korn Shell on Plan9 Jehanne

    Let start by saying that I’m not really a C programmer.
    My last public contribution to a POSIX C program was a little improvement to the Snort’s react module back in 2008.
    So while I know the C language well enough, I do not know anything about the subtleness of the standard library and I have little experience with POSIX semantics.
    This is not a big issue with Plan 9, since the C library and compiler are not standard anyway, but with Jehanne (a Plan 9 derivative of my own) I want to build a simple, loosely coupled, system that can actually run useful free software ported from UNIX.
    So I ported RedHat’s newlib to Jehanne on top of a new system library I wrote, LibPOSIX, that provides the necessary emulations. I wrote several test, checking they run the same on Linux and Jehanne, and then I begun looking for a real-world, battle tested, application to port first.
    I approached MirBSD’s Korn Shell for several reason:

    • it is simple, powerful and well written
    • it has been ported to several different operating systems
    • it has few dependencies
    • it’s the default shell in Android, so it’s really battle tested

    I was very confident. I had read the POSIX standard after all! And I had a test suite!
    I remember, I thought “Given newlib, how hard can it be?”
    The porting begun on September 1, 2017. It was completed by tg on January 5, 2018. 125 nights later.
    Turn out, my POSIX emulation was badly broken. Not just because of the usual bugs that any piece of C can have: I didn’t understood most POSIX semantics at all!

    iXsystems

    ###Static site generator with rsync and lowdown on OpenBSD

    • ssg is a tiny POSIX-compliant shell script with few dependencies:

    • lowdown(1) to parse markdown,

    • rsync(1) to copy temporary files, and

    • entr(1) to watch file changes.

    • It generates Markdown articles to a static website.

    • It copies the current directory to a temporary on in /tmp skipping .* and _*, renders all Markdown articles to HTML, generates RSS feed based on links from index.html, extracts the first <h1> tag from every article to generate a sitemap and use it as a page title, then wraps articles with a single HTML template, copies everything from the temporary directory to $DOCS/

    Why not Jekyll or “$X”?

    • ssg is one hundred times smaller than Jekyll.

    ssg and its dependencies are about 800KB combined. Compare that to 78MB of ruby with Jekyll and all the gems. So ssg can be installed in just few seconds on almost any Unix-like operating system.
    Obviously, ssg is tailored for my needs, it has all features I need and only those I use.
    Keeping ssg helps you to master your Unix-shell skills: awk, grep, sed, sh, cut, tr. As a web developer you work with lots of text: code and data. So you better master these wonderful tools.

    • Performance

    100 pps. On modern computers ssg generates a hundred pages per second. Half of a time for markdown rendering and another half for wrapping articles into the template. I heard good static site generators work—twice as fast—at 200 pps, so there’s lots of performance that can be gained. ;)

    ###Why does FreeBSD have virtually no (0%) desktop market share?

    • Because someone made a horrible design decision back in 1984.

    In absolute fairness to those involved, it was an understandable decision, both from a research perspective, and from an economic perspective, although likely not, from a technology perspective.

    • Why and what.

    The decision was taken because the X Window System was intended to run on cheap hardware, and, at the time, that meant reduced functionality in the end-point device with the physical display attached to it.
    At the same time, another force was acting to also limit X displays to display services only, rather than rolling in both window management and specific widget instances for common operational paradigms.
    Mostly, common operational paradigms didn’t really exist for windowing systems because they also simply didn’t exist at the time, and no one really knew how people were going to use the things, and so researchers didn’t want to commit future research to a set of hard constraints.
    So a decision was made: separate the display services from the application at the lowest level of graphics primitives currently in use at the time.

    • The ramifications of this were pretty staggering.

    First, it guaranteed that all higher level graphics would live on the host side of the X protocol, instead of on the display device side of the protocol.
    Despite a good understanding of Moore’s law, and the fact that, since no X Terminals existed at the time as hardware, but were instead running as emulations on workstations that had sufficient capability, this put the higher level GUI object libraries — referred to as “widgets” — in host libraries linked into the applications.
    Second, it guaranteed that display organization and management paradigms would also live on the host side of the protocol — assumed, in contradiction to the previous decision, to be running on the workstation.
    But, presumably, at some point, as lightweight X Terminals became available, to migrate to a particular host computer managing compute resource login/access services.

    • Between these early decisions reigned chaos.

    Specifically, the consequences of these decisions have been with us ever since:
    Look-and-feel are a consequence of the toolkit chosen by the application programmer, rather than a user decision which applies universally to all applications.
    You could call this “lack of a theme”, and — although I personally despise the idea of customizing or “theming” desktops — this meant that one paradigm chosen by the user would not apply universally across all applications, no matter who had written them.
    Window management style is a preference.
    You could call this a more radical version of “theming” — which you will remember, I despise — but a consequence to this is that training is not universal across personnel using such systems, nor is it transferrable.
    In other words, I can’t send someone to a class, and have them come back and use the computers in the office as a tool, with the computer itself — and the elements not specific to the application itself — disappearing into the background.
    Both of these ultimately render an X-based system unsuitable for desktops.
    I can’t pay once for training. Training that I do pay for does not easily and naturally translate between applications. Each new version may radically alter the desktop management paradigm into unrecognizability.

    • Is there hope for the future?

    Well, the Linux community has been working on something called Wayland, and it is very promising…
    …In the same way X was “very promising” in 1984, because, unfortunately, they are making exactly the same mistakes X made in 1984, rather than correcting them, now that we have 20/20 hindsight, and know what a mature widget library should look like.
    So Wayland is screwing up again.
    But hey, it only took us, what, 25 years to get from X in 1987 to Wayland in in 2012.
    Maybe if we try again in 2037, we can get to where Windows was in 1995.

    ##Beastie Bits

    Tarsnap

    ##Feedback/Questions

    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Thu, 05 Jul 2018 04:00:00 -0700
    Episode 252: Goes to 11.2 | BSD Now 252
    FreeBSD 11.2 has been released, setting up an MTA behind Tor, running pfsense on DigitalOcean, one year of C, using OpenBGPD to announce VM networks, the power to serve, and a BSDCan trip report.

    ##Headlines
    FreeBSD 11.2-RELEASE Available

    • FreeBSD 11.2 was released today (June 27th) and is ready for download
    • Highlights:

    OpenSSH has been updated to version 7.5p1.
    OpenSSL has been updated to version 1.0.2o.
    The clang, llvm, lldb and compiler-rt utilities have been updated to version 6.0.0.
    The libarchive(3) library has been updated to version 3.3.2.
    The libxo(3) library has been updated to version 0.9.0.
    Major Device driver updates to:

    • cxgbe(4) – Chelsio 10/25/40/50/100 gigabit NICs – version 1.16.63.0 supports T4, T5 and T6
    • ixl(4) – Intel 10 and 40 gigabit NICs, updated to version 1.9.9-k
    • ng_pppoe(4) – driver has been updated to add support for user-supplied Host-Uniq tags

    New drivers:
    + drm-next-kmod driver supporting integrated Intel graphics with the i915 driver.

    • mlx5io(4) – a new IOCTL interface for Mellanox ConnectX-4 and ConnectX-5 10/20/25/40/50/56/100 gigabit NICs
    • ocs_fc(4) – Emulex Fibre Channel 8/16/32 gigabit Host Adapters
    • smartpqi(4) – HP Gen10 Smart Array Controller Family

    The newsyslog(8) utility has been updated to support RFC5424-compliant messages when rotating system logs
    The diskinfo(8) utility has been updated to include two new flags, -s which displays the disk identity (usually the serial number), and -p which displays the physical path to the disk in a storage controller.
    The top(1) utility has been updated to allow filtering on multiple user names when the -U flag is used
    The umount(8) utility has been updated to include a new flag, -N, which is used to forcefully unmount an NFS mounted filesystem.
    The ps(1) utility has been updated to display if a process is running with capsicum(4) capability mode, indicated by the flag ‘C’
    The service(8) utility has been updated to include a new flag, -j, which is used to interact with services running within a jail(8). The argument to -j can be either the name or numeric jail ID
    The mlx5tool(8) utility has been added, which is used to manage Connect-X 4 and Connect-X 5 devices supported by mlx5io(4).
    The ifconfig(8) utility has been updated to include a random option, which when used with the ether option, generates a random MAC address for an interface.
    The dwatch(1) utility has been introduced
    The efibootmgr(8) utility has been added, which is used to manipulate the EFI boot manager.
    The etdump(1) utility has been added, which is used to view El Torito boot catalog information.
    The linux(4) ABI compatibility layer has been updated to include support for musl consumers.
    The fdescfs(5) filesystem has been updated to support Linux®-specific fd(4) /dev/fd and /proc/self/fd behavior
    Support for virtio_console(4) has been added to bhyve(4).
    The length of GELI passphrases entered when booting a system with encrypted disks is now hidden by default. See the configuration options in geli(8) to restore the previous behavior.

    • In addition to the usual CD/DVD ISO, Memstick, and prebuilt VM images (raw, qcow2, vhd, and vmdk), FreeBSD 11.2 is also available on:
      • Amazon EC2
      • Google Compute Engine
      • Hashicorp/Atlas Vagrant
      • Microsoft Azure
    • In addition to a generic ARM64 image for devices like the Pine64 and Raspberry Pi 3, specific images are provided for:
      • GUMSTIX
      • BANANAPI
      • BEAGLEBONE
      • CUBIEBOARD
      • CUBIEBOARD2
      • CUBOX-HUMMINGBOARD
      • RASPBERRY PI 2
      • PANDABOARD
      • WANDBOARD
    • Full Release Notes

    ###Setting up an MTA Behind Tor

    This article will document how to set up OpenSMTPD behind a fully Tor-ified network. Given that Tor’s DNS resolver code does not support MX record lookups, care must be taken for setting up an MTA behind a fully Tor-ified network. OpenSMTPD was chosen because it was easy to modify to force it to fall back to A/AAAA lookups when MX lookups failed with a DNS result code of NOTIMP (4).

    Note that as of 08 May 2018, the OpenSMTPD project is planning a configuration file language change. The proposed change has not landed. Once it does, this article will be updated to reflect both the old language and new.

    The reason to use an MTA behing a fully Tor-ified network is to be able to support email behind the .onion TLD. This setup will only allow us to send and receive email to and from the .onion TLD.

    • Requirements:

    • A fully Tor-ified network

    • HardenedBSD as the operating system

    • A server (or VM) running HardenedBSD behind the fully Tor-ified network.

    • /usr/ports is empty

    • Or is already pre-populated with the HardenedBSD Ports tree

    • Why use HardenedBSD? We get all the features of FreeBSD (ZFS, DTrace, bhyve, and jails) with enhanced security through exploit mitigations and system hardening. Tor has a very unique threat landscape and using a hardened ecosystem is crucial to mitigating risks and threats.

    Also note that this article reflects how I’ve set up my MTA. I’ve included configuration files verbatim. You will need to replace the text that refers to my .onion domain with yours.

    On 08 May 2018, HardenedBSD’s version of OpenSMTPD just gained support for running an MTA behind Tor. The package repositories do not yet contain the patch, so we will compile OpenSMTPD from ports.

    • Steps
    • Installation
    • Generating Cryptographic Key Material
    • Tor Configuration
    • OpenSMTPD Configuration
    • Dovecot Configuration
    • Testing your configuration
    • Optional: Webmail Access

    iXsystems
    https://www.forbes.com/sites/forbestechcouncil/2018/06/21/strings-attached-knowing-when-and-when-not-to-accept-vc-funding/#30f9f18f46ec
    https://www.ixsystems.com/blog/self-2018-recap/

    ###Running pfSense on a Digital Ocean Droplet

    I love pfSense (and opnSense, no discrimination here). I use it for just about anything, from homelab to large scale deployments and I’ll give out on any fancy <enter brand name fw appliance here> for a pfSense setup on a decent hardware.

    I also love DigitalOcean, if you ever used them, you know why, if you never did, head over and try, you’ll understand why.
    <shameless plug: head over to JupiterBroadcasting.com, the best technology content out there, they have coupon codes to get you started with DO>.

    Unfortunately, while DO offers tremendous amount of useful distros and applications, pfSense isn’t one of them. But, where there’s a will, there’s a way, and here’s how to get pfSense up and running on DO so you can have it as the gatekeeper to your kingdom.

    Start by creating a FreeBSD droplet, choose your droplet size (for modest setups, I find the 5$ to be quite awesome):

    There are many useful things you can do with pfSense on your droplet, from OpenVPN, squid, firewalling, fancy routing, url filtering, dns black listing and much much more.

    • One note though, before we wrap up:

    You have two ways to initiate the initial setup wizard of the web-configurator:
    Spin up another droplet, log into it and browse your way to the INTERNAL ip address of the internal NIC you’ve set up. This is the long and tedious way, but it’s also somewhat safer as it eliminates the small window of risk the second method poses.
    or
    Once your WAN address is all setup, your pfSense is ready to accept https connection to start the initial web-configurator setup.
    Thing is, there’s a default, well known set of credential to this initial wizard (admin:pfsense), so, there is a slight window of opportunity that someone can swoop in (assuming they know you’ve installed pfsense + your wan IP address + the exact time window between setting up the WAN interface and completing the wizard) and do <enter scary thing here>.

    I leave it up to you which of the path you’d like to go, either way, once you’re done with the web-configurator wizard, you’ll have a shiny new pfSense installation at your disposal running on your favorite VPS.

    Hopefully this was helpful for someone, I hope to get a similar post soon detailing how to get FreeNAS up and running on DO.
    Many thanks to Tubsta and his blogpost as well as to Allan Jude, Kris Moore and Benedict Reuschling for their AWESOME and inspiring podcast, BSD Now.

    ##News Roundup
    One year of C

    It’s now nearly a year that I started writing non-trivial amounts of C code again (the first sokol_gfx.h commit was on the 14-Jul-2017), so I guess it’s time for a little retrospective.

    In the beginning it was more of an experiment: I wanted to see how much I would miss some of the more useful C++ features (for instance namespaces, function overloading, ‘simple’ template code for containers, …), and whether it is possible to write non-trivial codebases in C without going mad.

    Here are all the github projects I wrote in C:

    • sokol: a slowly growing set of platform-abstraction headers
    • sokol-samples - examples for Sokol
    • chips - 8-bit chip emulators
    • chips-test - tests and examples for the chip- emulators, including some complete home computer emulators (minus sound)

    All in all these are around 32k lines of code (not including 3rd party code like flextGL and HandmadeMath). I think I wrote more C code in the recent 10 months than any other language.

    So one thing seems to be clear: yes, it’s possible to write a non-trivial amount of C code that does something useful without going mad (and it’s even quite enjoyable I might add).

    • Here’s a few things I learned:

    • Pick the right language for a problem

    • C is a perfect match for WebAssembly

    • C99 is a huge improvement over C89

    • The dangers of pointers and explicit memory management are overrated

    • Less Boilerplate Code

    • Less Language Feature ‘Anxiety’

    • Conclusion

    All in all my “C experiment” is a success. For a lot of problems, picking C over C++ may be the better choice since C is a much simpler language (btw, did you notice how there are hardly any books, conferences or discussions about C despite being a fairly popular language? Apart from the neverending bickering about undefined behaviour from the compiler people of course ;) There simply isn’t much to discuss about a language that can be learned in an afternoon.

    I don’t like some of the old POSIX or Linux APIs as much as the next guy (e.g. ioctl(), the socket API or some of the CRT library functions), but that’s an API design problem, not a language problem. It’s possible to build friendly C APIs with a bit of care and thinking, especially when C99’s designated initialization can be used (C++ should really make sure that the full C99 language can be used from inside C++ instead of continuing to wander off into an entirely different direction).

    ###Configuring OpenBGPD to announce VM’s virtual networks

    We use BGP quite heavily at work, and even though I’m not interacting with that directly, it feels like it’s something very useful to learn at least on some basic level. The most effective and fun way of learning technology is finding some practical application, so I decided to see if it could help to improve networking management for my Virtual Machines.

    My setup is fairly simple: I have a host that runs bhyve VMs and I have a desktop system from where I ssh to VMs, both hosts run FreeBSD. All VMs are connected to each other through a bridge and have a common network 10.0.1/24. The point of this exercise is to be able to ssh to these VMs from desktop without adding static routes and without adding vmhost’s external interfaces to the VMs bridge.

    I’ve installed openbgpd on both hosts and configured it like this:

    vmhost: /usr/local/etc/bgpd.conf AS 65002 router-id 192.168.87.48 fib-update no network 10.0.1.1/24 neighbor 192.168.87.41 { descr "desktop" remote-as 65001 }

    Here, router-id is set vmhost’s IP address in my home network (192.168.87/24), fib-update no is set to forbid routing table update, which I initially set for testing, but keeping it as vmhost is not supposed to learn new routes from desktop anyway. network announces my VMs network and neighbor describes my desktop box. Now the desktop box:

    desktop: /usr/local/etc/bgpd.conf AS 65001 router-id 192.168.87.41 fib-update yes neighbor 192.168.87.48 { descr "vmhost" remote-as 65002 }

    It’s pretty similar to vmhost’s bgpd.conf, but no networks are announced here, and fib-update is set to yes because the whole point is to get VM routes added. Both hosts have to have the openbgpd service enabled:

    /etc/rc.conf.local openbgpdenable="YES"
    • Conclusion

    As mentioned already, similar result could be achieved without using BGP by using either static routes or bridging interfaces differently, but the purpose of this exercise is to get some basic hands-on experience with BGP. Right now I’m looking into extending my setup in order to try more complex BGP schema. I’m thinking about adding some software switches in front of my VMs or maybe adding a second VM host (if budget allows). You’re welcome to comment if you have some ideas how to extend this setup for educational purposes in the context of BGP and networking.

    As a side note, I really like openbgpd so far. Its configuration file format is clean and simple, documentation is good, error and information messages are clear, and CLI has intuitive syntax.

    Digital Ocean

    ###The Power to Serve

    All people within the IT Industry should known where the slogan “The Power To Serve” is exposed every day to millions of people. But maybe too much wishful thinking from me. But without “The Power To Serve” the IT industry today will look totally different. Companies like Apple, Juniper, Cisco and even WatsApp would not exist in their current form.

    I provide IT architecture services to make your complex IT landscape manageable and I love to solve complex security and privacy challenges. Complex challenges where people, processes and systems are heavily interrelated. For this knowledge intensive work I often run some IT experiments. When you run experiments nowadays you have a choice:

    • Rent some cloud based services or
    • DIY (Do IT Yourself) on premise

    Running your own developments experiments on your own infrastructure can be time consuming. However smart automation saves time and money. And by creating your own CICD pipeline (Continuous Integration, Continuous Deployment) you stay on top of core infrastructure developments. Even hands-on. Knowing how things work from a technical ‘hands-on’ perspective gives great advantages when it comes to solving complex business IT problems. Making a clear distinguish between a business problem or IT problem is useless. Business and IT problems are related. Sometimes causal related, but more often indirect by one or more non linear feedback loops. Almost every business depends of IT systems. Bad IT means often that your customers will leave your business.

    One of the things of FeeBSD for me is still FreeBSD Jails. In 2015 I had luck to attend to a presentation of the legendary hacker Poul-Henning Kamp . Check his BSD bio to see what he has done for the FreeBSD community! FreeBSD jails are a light way to visualize your system without enormous overhead. Now that the development on Linux for LXD/LXD is more mature (lxd is the next generation system container manager on linux) there is finally again an alternative for a nice chroot Linux based system again. At least when you do not need the overhead and management complexity that comes with Kubernetes or Docker.

    FreeBSD means control and quality for me. When there is an open source package I need, I want to install it from source. It gives me more control and always some extra knowledge on how things work. So no precompiled binaries for me on my BSD systems! If a build on FreeBSD fails most of the time this is an alert regarding the quality for me.

    If a complex OSS package is not available at all in the FreeBSD ports collection there should be a reason for it. Is it really that nobody on the world wants to do this dirty maintenance work? Or is there another cause that running this software on FreeBSD is not possible…There are currently 32644 ports available on FreeBSD. So all the major programming language, databases and middleware libraries are present. The FreeBSD organization is a mature organization and since this is one of the largest OSS projects worldwide learning how this community manages to keep innovation and creates and maintains software is a good entrance for learning how complex IT systems function.

    FreeBSD is of course BSD licensed. It worked well! There is still a strong community with lots of strong commercial sponsors around the community. Of course: sometimes a GPL license makes more sense. So beside FreeBSD I also love GPL software and the rationale and principles behind it. So my hope is that maybe within the next 25 years the hard battle between BSD vs GPL churches will be more rationalized and normalized. Principles are good, but as all good IT architects know: With good principles alone you never make a good system. So use requirements and not only principles to figure out what OSS license fits your project. There is never one size fits all.

    June 19, 1993 was the day the official name for FreeBSD was agreed upon. So this blog is written to celebrate 25th anniversary of FreeBSD.

    ###Dave’s BSDCan trip report

    • So far, only one person has bothered to send in a BSDCan trip report. Our warmest thanks to Dave for doing his part.

    Hello guys! During the last show, you asked for a trip report regarding BSDCan 2018.
    This was my first time attending BSDCan. However, BSDCan was my second BSD conference overall, my first being vBSDCon 2017 in Reston, VA.
    Arriving early Thursday evening and after checking into the hotel, I headed straight to the Red Lion for the registration, picked up my badge and swag and then headed towards the ‘DMS’ building for the newbies talk. The only thing is, I couldn’t find the DMS building! Fortunately I found a BSDCan veteran who was heading there themselves. My only suggestion is to include the full building name and address on the BSDCan web site, or even a link to Google maps to help out with the navigation. The on-campus street maps didn’t have ‘DMS’ written on them anywhere. But I digress.
    Once I made it to the newbies talk hosted by Dan Langille and Michael W Lucas, it highlighted places to meet, an overview of what is happening, details about the ‘BSDCan widow/widower tours’ and most importantly, the 6-2-1 rule!
    The following morning, we were present with tea/coffee, muffins and other goodies to help prepare us for the day ahead.
    The first talk, “The Tragedy of systemd” covered what systemd did wrong and how the BSD community could improve on the ideas behind it.
    With the exception of Michael W Lucas, SSH Key Management and Kirk McKusick, The Evolution of FreeBSD Governance talk, I pretty much attended all of the ZFS talks including the lunchtime BoF session, hosted by Allan Jude. Coming from FreeNAS and being involved in the community, this is where my main interest and motivation lies. Since then I have been able to share some of that information with the FreeNAS community forums and chatroom.
    I also attended the “Speculating about Intel” lunchtime BoF session hosted by Theo de Raddt, which proved to be “interesting”.
    The talks ended with the wrap up session with a few words from Dan, covering the record attendance and made very clear there “was no cabal”. Followed by the the handing over of Groff the BSD goat to a new owner, thank you’s from the FreeBSD Foundation to various community committers and maintainers, finally ending with the charity auction, where a things like a Canadian $20 bill sold for $40, a signed FreeBSD Foundation shirt originally worn by George Neville-Neil, a lost laptop charger, Michael’s used gelato spoon, various books, the last cookie and more importantly, the second to last cookie!
    After the auction, we all headed to the Red Lion for food and drinks, sponsored by iXsystems.
    I would like to thank the BSDCan organizers, speakers and sponsors for a great conference. I will certainly hope to attend next year!
    Regards,
    Dave (aka m0nkey)

    • Thanks to Dave for sharing his experiences with us and our viewers

    ##Beastie Bits

    Tarsnap

    ##Feedback/Questions

    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Thu, 28 Jun 2018 00:00:00 -0700
    Episode 251: Crypto HAMMER | BSD Now 251
    DragonflyBSD’s hammer1 encrypted master/slave setup, second part of our BSDCan recap, NomadBSD 1.1-RC1 available, OpenBSD adds an LDAP client to base, FreeBSD gets pNFS support, Intel FPU Speculation Vulnerability confirmed, and what some Unix command names mean.

    ##Headlines
    DragonflyBSD: Towards a HAMMER1 master/slave encrypted setup with LUKS

    I just wanted to share my experience with setting up DragonFly master/slave HAMMER1 PFS’s on top of LUKS
    So after a long time using an Synology for my NFS needs, I decided it was time to rethink my setup a little since I had several issues with it :

    • You cannot run NFS on top of encrypted partitions easily
    • I suspect I am having some some data corruption (bitrot) on the ext4 filesystem
    • the NIC was stcuk to 100 Mbps instead of 1 Gbps even after swapping cables, switches, you name it
    • It’s proprietary

    I have been playing with DragonFly in the past and knew about HAMMER, now I just had the perfect excuse to actually use it in production :) After setting up the OS, creating the LUKS partition and HAMMER FS was easy :

    kdload dm
    cryptsetup luksFormat /dev/serno/<id1>
    cryptsetup luksOpen /dev/serno/<id1> fort_knox
    newfs_hammer -L hammer1_secure_master /dev/mapper/fort_knox
    cryptsetup luksFormat /dev/serno/<id2>
    cryptsetup luksOpen /dev/serno/<id2> fort_knox_slave
    newfs_hammer -L hammer1_secure_slave /dev/mapper/fort_knox_slave

    • Mount the 2 drives :

    mount /dev/mapper/fort_knox /fort_knox
    mount /dev/mapper_fort_know_slave /fort_knox_slave

    You can now put your data under /fort_knox
    Now, off to setting up the replication, first get the shared-uuid of /fort_knox

    hammer pfs-status /fort_knox

    Create a PFS slave “linked” to the master

    hammer pfs-slave /fort_knox_slave/pfs/slave shared-uuid=f9e7cc0d-eb59-10e3-a5b5-01e6e7cefc12

    And then stream your data to the slave PFS !

    hammer mirror-stream /fort_knox /fort_knox_slave/pfs/slave

    After that, setting NFS is fairly trivial even though I had problem with the /etc/exports syntax which is different than Linux

    There’s a few things I wish would be better though but nothing too problematic or without workarounds :

    • Cannot unlock LUKS partitions at boot time afaik (Acceptable tradeoff for the added security LUKS gives me vs my old Synology setup) but this force me to run a script to unlock LUKS, mount hammer and start mirror-stream at each boot
    • No S1/S3 sleep so I made a script to shutdown the system when there’s no network neighborgs to serve the NFS
    • As my system isn’t online 24/7 for energy reasons, I guess will have to run hammer cleanup myself from time to time
    • Some uncertainty because hey, it’s kind of exotic but exciting too :)

    Overall, I am happy, HAMMER1 and PFS are looking really good, DragonFly is a neat Unix and the community is super friendly (Matthew Dillon actually provided me with a kernel patch to fix the broken ACPI on the PC holding this setup, many thanks!), the system is still a “work in progress” but it is already serving my files as I write this post.

    Let’s see in 6 months how it goes in the longer run !

    ###BSDCan 2018 Recap

    • As promised, here is our second part of our BSDCan report, covering the conference proper. The last tutorials/devsummit of that day lead directly into the conference, as people could pick up their registration packs at the Red Lion and have a drink with fellow BSD folks.
    • Allan and I were there only briefly, as we wanted to get back to the “Newcomers orientation and mentorship” session lead by Michael W. Lucas. This session is intended for people that are new to BSDCan (maybe their first BSD conference ever?) and may have questions. Michael explained everything from the 6-2-1 rule (hours of sleep, meals per day, and number of showers that attendees should have at a minimum), to the partner and widowers program (lead by his wife Liz), to the sessions that people should not miss (opening, closing, and hallway track). Old-time BSDCan folks were asked to stand up so that people can recognize them and ask them any questions they might have during the conferences. The session was well attended. Afterwards, people went for dinner in groups, a big one lead by Michael Lucas to his favorite Shawarma place, followed by gelato (of course). This allowed newbies to mingle over dinner and ice cream, creating a welcoming atmosphere.
    • The next day, after Dan Langille opened the conference, Benno Rice gave the keynote presentation about “The Tragedy of Systemd”.
    • Benedict went to the following talks:

    “Automating Network Infrastructures with Ansible on FreeBSD” in the DevSummit track. A good talk that connected well with his Ansible tutorial and even allowed some discussions among participants.
    “All along the dwatch tower”: Devin delivered a well prepared talk. I first thought that the number of slides would not fit into the time slot, but she even managed to give a demo of her work, which was well received. The dwatch tool she wrote should make it easy for people to get started with DTrace without learning too much about the syntax at first. The visualizations were certainly nice to see, combining different tools together in a new way.
    ZFS BoF, lead by Allan and Matthew Ahrens
    SSH Key Management by Michael W. Lucas. Yet another great talk where I learned a lot. I did not get to the SSH CA chapter in the new SSH Mastery book, so this was a good way to wet my appetite for it and motivated me to look into creating one for the cluster that I’m managing.
    The rest of the day was spent at the FreeBSD Foundation table, talking to various folks. Then, Allan and I had an interview with Kirk McKusick for National FreeBSD Day, then we had a core meeting, followed by a core dinner.

    • Day 2:

      “Flexible Disk Use in OpenZFS”: Matthew Ahrens talking about the feature he is implementing to expand a RAID-Z with a single disk, as well as device removal.
      Allan’s talk about his efforts to implement ZSTD in OpenZFS as another compression algorithm. I liked his overview slides with the numbers comparing the algorithms for their effectiveness and his personal story about the sometimes rocky road to get the feature implemented.
      “zrepl - ZFS replication” by Christian Schwarz, was well prepared and even had a demo to show what his snapshot replication tool can do. We covered it on the show before and people can find it under sysutils/zrepl. Feedback and help is welcome.
      “The Evolution of FreeBSD Governance” by Kirk McKusick was yet another great talk by him covering the early days of FreeBSD until today, detailing some of the progress and challenges the project faced over the years in terms of leadership and governance. This is an ongoing process that everyone in the community should participate in to keep the project healthy and infused with fresh blood.
      Closing session and auction were funny and great as always.
      All in all, yet another amazing BSDCan. Thank you Dan Langille and your organizing team for making it happen! Well done.

    Digital Ocean

    ###NomadBSD 1.1-RC1 Released

    The first – and hopefully final – release candidate of NomadBSD 1.1 is available!

    • Changes
    • The base system has been upgraded to FreeBSD 11.2-RC3
    • EFI booting has been fixed.
    • Support for modern Intel GPUs has been added.
    • Support for installing packages has been added.
    • Improved setup menu.
    • More software packages:
    • benchmarks/bonnie++
    • DSBDisplaySettings
    • DSBExec
    • DSBSu
    • mail/thunderbird
    • net/mosh
    • ports-mgmt/octopkg
    • print/qpdfview
    • security/nmap
    • sysutils/ddrescue
    • sysutils/fusefs-hfsfuse
    • sysutils/fusefs-sshfs
    • sysutils/sleuthkit
    • www/lynx
    • x11-wm/compton
    • x11/xev
    • x11/xterm
    • Many improvements and bugfixes
      The image and instructions can be found here.

    ##News Roundup
    LDAP client added to -current

    CVSROOT: /cvs Module name: src Changes by: reyk@cvs.openbsd.org 2018/06/13 09:45:58 Log message: Import ldap(1), a simple ldap search client. We have an ldapd(8) server and ypldap in base, so it makes sense to have a simple LDAP client without depending on the OpenLDAP package. This tool can be used in an ssh(1) AuthorizedKeysCommand script. With feedback from many including millert@ schwarze@ gilles@ dlg@ jsing@ OK deraadt@ Status: Vendor Tag: reyk Release Tags: ldap_20180613 N src/usr.bin/ldap/Makefile N src/usr.bin/ldap/aldap.c N src/usr.bin/ldap/aldap.h N src/usr.bin/ldap/ber.c N src/usr.bin/ldap/ber.h N src/usr.bin/ldap/ldap.1 N src/usr.bin/ldap/ldapclient.c N src/usr.bin/ldap/log.c N src/usr.bin/ldap/log.h No conflicts created by this import

    ###Intel® FPU Speculation Vulnerability Confirmed

    • Earlier this month, Philip Guenther (guenther@) committed (to amd64 -current) a change from lazy to semi-eager FPU switching to mitigate against rumored FPU state leakage in Intel® CPUs.
    • Theo de Raadt (deraadt@) discussed this in his BSDCan 2018 session.
    • Using information disclosed in Theo’s talk, Colin Percival developed a proof-of-concept exploit in around 5 hours. This seems to have prompted an early end to an embargo (in which OpenBSD was not involved), and the official announcement of the vulnerability.
    • FPU change in FreeBSD
    Summary: System software may utilize the Lazy FP state restore technique to delay the restoring of state until an instruction operating on that state is actually executed by the new process. Systems using Intel® Core-based microprocessors may potentially allow a local process to infer data utilizing Lazy FP state restore from another process through a speculative execution side channel. Description: System software may opt to utilize Lazy FP state restore instead of eager save and restore of the state upon a context switch. Lazy restored states are potentially vulnerable to exploits where one process may infer register values of other processes through a speculative execution side channel that infers their value. · CVSS - 4.3 Medium CVSS:3.0/AV:L/AC:L/PR:N/UI:N/S:C/C:L/I:N/A:N Affected Products: Intel® Core-based microprocessors. Recommendations: If an XSAVE-enabled feature is disabled, then we recommend either its state component bitmap in the extended control register (XCR0) is set to 0 (e.g. XCR0[bit 2]=0 for AVX, XCR0[bits 7:5]=0 for AVX512) or the corresponding register states of the feature should be cleared prior to being disabled. Also for relevant states (e.g. x87, SSE, AVX, etc.), Intel recommends system software developers utilize Eager FP state restore in lieu of Lazy FP state restore. Acknowledgements: Intel would like to thank Julian Stecklina from Amazon Germany, Thomas Prescher from Cyberus Technology GmbH (https://www.cyberus-technology.de/), Zdenek Sojka from SYSGO AG (http://sysgo.com), and Colin Percival for reporting this issue and working with us on coordinated disclosure.

    iXsystems
    iX Ad Spot
    iX Systems - BSDCan 2018 Recap

    ###FreeBSD gets pNFS support

    Merge the pNFS server code from projects/pnfs-planb-server into head. This code merge adds a pNFS service to the NFSv4.1 server. Although it is a large commit it should not affect behaviour for a non-pNFS NFS server. Some documentation on how this works can be found at: Merge the pN http://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt and will hopefully be turned into a proper document soon. This is a merge of the kernel code. Userland and man page changes will come soon, once the dust settles on this merge. It has passed a "make universe", so I hope it will not cause build problems. It also adds NFSv4.1 server support for the "current stateid". Here is a brief overview of the pNFS service: A pNFS service separates the Read/Write operations from all the other NFSv4.1 Metadata operations. It is hoped that this separation allows a pNFS service to be configured that exceeds the limits of a single NFS server for either storage capacity and/or I/O bandwidth. It is possible to configure mirroring within the data servers (DSs) so that the data storage file for an MDS file will be mirrored on two or more of the DSs. When this is used, failure of a DS will not stop the pNFS service and a failed DS can be recovered once repaired while the pNFS service continues to operate. Although two way mirroring would be the norm, it is possible to set a mirroring level of up to four or the number of DSs, whichever is less. The Metadata server will always be a single point of failure, just as a single NFS server is. A Plan B pNFS service consists of a single MetaData Server (MDS) and K Data Servers (DS), all of which are recent FreeBSD systems. Clients will mount the MDS as they would a single NFS server. When files are created, the MDS creates a file tree identical to what a single NFS server creates, except that all the regular (VREG) files will be empty. As such, if you look at the exported tree on the MDS directly on the MDS server (not via an NFS mount), the files will all be of size 0. Each of these files will also have two extended attributes in the system attribute name space: pnfsd.dsfile - This extended attrbute stores the information that the MDS needs to find the data storage file(s) on DS(s) for this file. pnfsd.dsattr - This extended attribute stores the Size, AccessTime, ModifyTime and Change attributes for the file, so that the MDS doesn't need to acquire the attributes from the DS for every Getattr operation. For each regular (VREG) file, the MDS creates a data storage file on one (or more if mirroring is enabled) of the DSs in one of the "dsNN" subdirectories. The name of this file is the file handle of the file on the MDS in hexadecimal so that the name is unique. The DSs use subdirectories named "ds0" to "dsN" so that no one directory gets too large. The value of "N" is set via the sysctl vfs.nfsd.dsdirsize on the MDS, with the default being 20. For production servers that will store a lot of files, this value should probably be much larger. It can be increased when the "nfsd" daemon is not running on the MDS, once the "dsK" directories are created. For pNFS aware NFSv4.1 clients, the FreeBSD server will return two pieces of information to the client that allows it to do I/O directly to the DS. DeviceInfo - This is relatively static information that defines what a DS is. The critical bits of information returned by the FreeBSD server is the IP address of the DS and, for the Flexible File layout, that NFSv4.1 is to be used and that it is "tightly coupled". There is a "deviceid" which identifies the DeviceInfo. Layout - This is per file and can be recalled by the server when it is no longer valid. For the FreeBSD server, there is support for two types of layout, call File and Flexible File layout. Both allow the client to do I/O on the DS via NFSv4.1 I/O operations. The Flexible File layout is a more recent variant that allows specification of mirrors, where the client is expected to do writes to all mirrors to maintain them in a consistent state. The Flexible File layout also allows the client to report I/O errors for a DS back to the MDS. The Flexible File layout supports two variants referred to as "tightly coupled" vs "loosely coupled". The FreeBSD server always uses the "tightly coupled" variant where the client uses the same credentials to do I/O on the DS as it would on the MDS. For the "loosely coupled" variant, the layout specifies a synthetic user/group that the client uses to do I/O on the DS. The FreeBSD server does not do striping and always returns layouts for the entire file. The critical information in a layout is Read vs Read/Writea and DeviceID(s) that identify which DS(s) the data is stored on. At this time, the MDS generates File Layout layouts to NFSv4.1 clients that know how to do pNFS for the non-mirrored DS case unless the sysctl vfs.nfsd.default_flexfile is set non-zero, in which case Flexible File layouts are generated. The mirrored DS configuration always generates Flexible File layouts. For NFS clients that do not support NFSv4.1 pNFS, all I/O operations are done against the MDS which acts as a proxy for the appropriate DS(s). When the MDS receives an I/O RPC, it will do the RPC on the DS as a proxy. If the DS is on the same machine, the MDS/DS will do the RPC on the DS as a proxy and so on, until the machine runs out of some resource, such as session slots or mbufs. As such, DSs must be separate systems from the MDS. *** ###[What does {some strange unix command name} stand for?](http://www.unixguide.net/unix/faq/1.3.shtml) + awk = "Aho Weinberger and Kernighan" + grep = "Global Regular Expression Print" + fgrep = "Fixed GREP". + egrep = "Extended GREP" + cat = "CATenate" + gecos = "General Electric Comprehensive Operating Supervisor" + nroff = "New ROFF" + troff = "Typesetter new ROFF" + tee = T + bss = "Block Started by Symbol + biff = "BIFF" + rc (as in ".cshrc" or "/etc/rc") = "RunCom" + Don Libes' book "Life with Unix" contains lots more of these tidbits. *** ##Beastie Bits + [RetroBSD: Unix for microcontrollers](http://retrobsd.org/wiki/doku.php) + [On the matter of OpenBSD breaking embargos (KRACK)](https://marc.info/?l=openbsd-tech&m=152910536208954&w=2) + [Theo's Basement Computer Paradise (1998)](https://zeus.theos.com/deraadt/hosts.html) + [Airport Extreme runs NetBSD](https://jcs.org/2018/06/12/airport_ssh) + [What UNIX shell could have been](https://rain-1.github.io/shell-2.html) *** Tarsnap ad *** ##Feedback/Questions + We need more feedback and questions. Please email feedback@bsdnow.tv + Also, many of you owe us BSDCan trip reports! We have shared what our experience at BSDCan was like, but we want to hear about yours. What can we do better next year? What was it like being there for the first time? + [Jason writes in](https://slexy.org/view/s205jU58X2) + https://www.wheelsystems.com/en/products/wheel-fudo-psm/ + [June 19th was National FreeBSD Day](https://twitter.com/search?src=typd&q=%23FreeBSDDay) *** - Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [feedback@bsdnow.tv](mailto:feedback@bsdnow.tv) ***
    Thu, 21 Jun 2018 02:00:00 -0700
    Episode 250: BSDCan 2018 Recap | BSD Now 250
    TrueOS becoming a downstream fork with Trident, our BSDCan 2018 recap, HardenedBSD Foundation founding efforts, VPN with OpenIKED on OpenBSD, FreeBSD on a System76 Galago Pro, and hardware accelerated crypto on Octeons.

    ##Headlines##
    TrueOS to Focus on Core Operating System

    The TrueOS Project has some big plans in the works, and we want to take a minute and share them with you. Many have come to know TrueOS as the “graphical FreeBSD” that makes things easy for newcomers to the BSDs. Today we’re announcing that TrueOS is shifting our focus a bit to become a cutting-edge operating system that keeps all of the stability that you know and love from ZFS (OpenZFS) and FreeBSD, and adds additional features to create a fresh, innovative operating system. Our goal is to create a core-centric operating system that is modular, functional, and perfect for do-it-yourselfers and advanced users alike.

    TrueOS will become a downstream fork that will build on FreeBSD by integrating new software technologies like OpenRC and LibreSSL. Work has already begun which allows TrueOS to be used as a base platform for other projects, including JSON-based manifests, integrated Poudriere / pkg tools and much more. We’re planning on a six month release cycle to keep development moving and fresh, allowing us to bring you hot new features to ZFS, bhyve and related tools in a timely manner. This makes TrueOS the perfect fit to serve as the basis for building other distributions.

    Some of you are probably asking yourselves “But what if I want to have a graphical desktop?” Don’t worry! We’re making sure that everyone who knows and loves the legacy desktop version of TrueOS will be able to continue using a FreeBSD-based, graphical operating system in the future. For instance, if you want to add KDE, just use sudo pkg install kde and voila! You have your new shiny desktop. Easy right? This allows us to get back to our roots of being a desktop agnostic operating system. If you want to add a new desktop environment, you get to pick the one that best suits your use.

    We know that some of you will still be looking for an out-of-the-box solution similar to legacy PC-BSD and TrueOS. We’re happy to announce that Project Trident will take over graphical FreeBSD development going forward. Not much is going to change in that regard other than a new name! You’ll still have Lumina Desktop as a lightweight and feature-rich desktop environment and tons of utilities from the legacy TrueOS toolchain like sysadm and AppCafe. There will be migration paths available for those that would like to move to other FreeBSD-based distributions like Project Trident or GhostBSD.

    We look forward to this new chapter for TrueOS and hope you will give the new edition a spin! Tell us what you think about the new changes by leaving us a comment. Don’t forget you can ask us questions on our Twitter and be a part of our community by joining the new TrueOS Forums when they go live in about a week. Thanks for being a loyal fan of TrueOS.

    ###Project Trident FAQ

    • Q: Why did you pick the name “Project Trident”?

    A: We were looking for a name that was unique, yet would still relate to the BSD community. Since Beastie (the FreeBSD mascot) is always pictured with a trident, it felt like that would be a great name.

    • Q: Where can users go for technical support?

    A: At the moment, Project Trident will continue sharing the TrueOS community forums and Telegram channels. We are currently evaluating dedicated options for support channels in the future.

    • Q: Can I help contribute to the project?

    A: We are always looking for developers who want to join the project. If you’re not a developer you can still help, as a community project we will be more reliant on contributions from the community in the form of how-to guides and other user-centric documentation and support systems.

    • Q: How is the project supported financially?

    A: Project Trident is sponsored by the community, from both individuals and corporations. iXsystems has stepped up as the first enterprise-level sponsor of the project, and has been instrumental in getting Project Trident up and running. Please visit the Sponsors page to see all the current sponsors.

    • Q: How can I help support the project financially?

    A: Several methods exist, from one time or recurring donations via Paypal to limited time swag t-shirt campaigns during the year. We are also looking into more alternative methods of support, so please visit the Sponsors page to see all the current methods of sponsorship.

    • Q: Will there be any transparency of the financial donations and expenditures?

    A: Yes, we will be totally open with how much money comes into the project and what it is spent on. Due to concerns of privacy, we will not identify individuals and their donation amounts unless they specifically request to be identified. We will release a monthly overview in/out ledger, so that community members can see where their money is going.

    • Relationship with TrueOS

    • Project Trident does have very close ties to the TrueOS project, since most of the original Project Trident developers were once part of the TrueOS project before it became a distribution platform. For users of the TrueOS desktop, we have some additional questions and answers below.

    • Q: Do we need to be at a certain TrueOS install level/release to upgrade?

    A: As long as you have a TrueOS system which has been updated to at least the 18.03 release you should be able to just perform a system update to be automatically upgraded to Project Trident.

    • Q: Which members moved from TrueOS to Project Trident?

    A: Project Trident is being led by prior members of the TrueOS desktop team. Ken and JT (development), Tim (documentation) and Rod (Community/Support). Since Project Trident is a community-first project, we look forward to working with new members of the team.

    iXsystems

    ###BSDCan

    • BSDCan finished Saturday last week
    • It started with the GoatBoF on Tuesday at the Royal Oak Pub, where people had a chance to meet and greet. Benedict could not attend due to an all-day FreeBSD Foundation meeting and and even FreeBSD Journal Editorial Board meeting.
    • The FreeBSD devsummit was held the next two days in parallel to the tutorials. Gordon Tetlow, who organized the devsummit, opened the devsummit. Deb Goodkin from the FreeBSD Foundation gave the first talk with a Foundation update, highlighting current and future efforts. Li-Wen Hsu is now employed by the Foundation to assist in QA work (Jenkins, CI/CD) and Gordon Tetlow has a part-time contract to help secteam as their secretary.
    • Next, the FreeBSD core team (among them Allan and Benedict) gave a talk about what has happened this last term. With a core election currently running, some of these items will carry over to the next core team, but there were also some finished ones like the FCP process and FreeBSD members initiative. People in the audience asked questions on various topics of interest.
    • After the coffee break, the release engineering team gave a talk about their efforts in terms of making releases happen in time and good quality.
    • Benedict had to give his Ansible tutorial in the afternoon, which had roughly 15 people attending. Most of them beginners, we could get some good discussions going and I also learned a few new tricks. The overall feedback was positive and one even asked what I’m going to teach next year.
    • The second day of the FreeBSD devsummit began with Gordon Tetlow giving an insight into the FreeBSD Security team (aka secteam). He gave a overview of secteam members and responsibilities, explaining the process based on a long past advisory. Developers were encouraged to help out secteam. NDAs and proper disclosure of vulnerabilities were also discussed, and the audience had some feedback and questions.
    • When the coffee break was over, the FreeBSD 12.0 planning session happened. A Google doc served as a collaborative way of gathering features and things left to do. People signed up for it or were volunteered. Some features won’t make it into 12.0 as they are not 100% ready for prime time and need a few more rounds of testing and bugfixing. Still, 12.0 will have some compelling features.
    • A 360° group picture was taken after lunch, and then people split up into the working groups for the afternoon or started hacking in the UofO Henderson residence.
    • Benedict and Allan both attended the OpenZFS working group, lead by Matt Ahrens. He presented the completed and outstanding work in FreeBSD, without spoiling too much of the ZFS presentations of various people that happened later at the conference.
    • Benedict joined the boot code session a bit late (hallway track is the reason) when most things seem to have already been discussed.
    • BSDCan 2018 — Ottawa (In Pictures)
    • iXsystems Photos from BSDCan 2018

    ##News Roundup
    June HardenedBSD Foundation Update

    We at HardenedBSD are working towards starting up a 501©(3) not-for-profit organization in the USA. Setting up this organization will allow future donations to be tax deductible. We’ve made progress and would like to share with you the current state of affairs.

    We have identified, sent invitations out, and received acceptance letters from six people who will serve on the HardenedBSD Foundation Board of Directors. You can find their bios below. In the latter half of June 2018 or the beginning half of July 2018, we will meet for the first time as a board and formally begin the process of creating the documentation needed to submit to the local, state, and federal tax services.

    Here’s a brief introduction to those who will serve on the board:

    • W. Dean Freeman (Advisor): Dean has ten years of professional experience with deploying and security Unix and networking systems, including assessing systems security for government certification and assessing the efficacy of security products. He was introduced to Unix via FreeBSD 2.2.8 on an ISP shell account as a teenager. Formerly, he was the Snort port maintainer for FreeBSD while working in the Sourcefire VRT, and has contributed entropy-related patches to the FreeBSD and HardenedBSD projects – a topic on which he presented at vBSDCon 2017.

    • Ben La Monica (Advisor): Ben is a Senior Technology Manager of Software Engineering at Morningstar, Inc and has been developing software for over 15 years in a variety of languages. He advocates open source software and enjoys tinkering with electronics and home automation.

    • George Saylor (Advisor): George is a Technical Directory at G2, Inc. Mr. Saylor has over 28 years of information systems and security experience in a broad range of disciplines. His core focus areas are automation and standards in the event correlation space as well as penetration and exploitation of computer systems. Mr Saylor was also a co-founder of the OpenSCAP project.

    • Virginia Suydan (Accountant and general administrator): Accountant and general administrator for the HardenedBSD Foundation. She has worked with Shawn Webb for tax and accounting purposes for over six years.

    • Shawn Webb (Director): Co-founder of HardenedBSD and all-around infosec wonk. He has worked and played in the infosec industry, doing both offensive and defensive research, for around fifteen years. He loves open source technologies and likes to frustrate the bad guys.

    • Ben Welch (Advisor): Ben is currently a Security Engineer at G2, Inc. He graduated from Pennsylvania College of Technology with a Bachelors in Information Assurance and Security. Ben likes long walks, beaches, candlelight dinners, and attending various conferences like BSides and ShmooCon.

    ###Your own VPN with OpenIKED & OpenBSD

    Remote connectivity to your home network is something I think a lot of people find desirable. Over the years, I’ve just established an SSH tunnel and use it as a SOCKS proxy, sending my traffic through that. It’s a nice solution for a “poor man’s VPN”, but it can be a bit clunky, and it’s not great having to expose SSH to the world, even if you make sure to lock everything down

    I set out the other day to finally do it properly. I’d come across this great post by Gordon Turner: OpenBSD 6.2 VPN Endpoint for iOS and macOS

    Whilst it was exactly what I was looking for, it outlined how to set up an L2TP VPN. Really, I wanted IKEv2 for performance and security reasons (I won’t elaborate on this here, if you’re curious about the differences, there’s a lot of content out on the web explaining this).

    The client systems I’d be using have native support for IKEv2 (iOS, macOS, other BSD systems). But, I couldn’t find any tutorials in the same vein.

    So, let’s get stuck in!

    • A quick note ✍️

    This guide will walk through the set up of an IKEv2 VPN using OpenIKED on OpenBSD. It will detail a “road warrior” configuration, and use a PSK (pre-shared-key) for authentication. I’m sure it can be easily adapted to work on any other platforms that OpenIKED is available on, but keep in mind my steps are specifically for OpenBSD.

    • Server Configuration

    As with all my home infrastructure, I crafted this set-up declaratively. So, I had the deployment of the VM setup in Terraform (deployed on my private Triton cluster), and wrote the configuration in Ansible, then tied them together using radekg/terraform-provisioner-ansible.

    One of the reasons I love Ansible is that its syntax is very simplistic, yet expressive. As such, I feel it fits very well into explaining these steps with snippets of the playbook I wrote. I’ll link the full playbook a bit further down for those interested.

    • See the full article for the information on:
    • sysctl parameters
    • The naughty list (optional)
    • Configure the VPN network interface
    • Configure the firewall
    • Configure the iked service
    • Gateway configuration
    • Client configuration
    • Troubleshooting

    DigitalOcean

    ###FreeBSD on a System76 Galago Pro

    Hey all, It’s been a while since I last posted but I thought I would hammer something out here. My most recent purchase was a System76 Galago Pro. I thought, afer playing with POP! OS a bit, is there any reason I couldn’t get BSD on this thing. Turns out the answer is no, no there isnt and it works pretty decently.

    To get some accounting stuff out of the way I tested this all on FreeBSD Head and 11.1, and all of it is valid as of May 10, 2018. Head is a fast moving target so some of this is only bound to improve.

    • The hardware

    • Intel Core i5 Gen 8

    • UHD Graphics 620

    • 16 GB DDR4 Ram

    • RTL8411B PCI Express Card Reader

    • RTL8111 Gigabit ethernet controller

    • Intel HD Audio

    • Samsung SSD 960 PRO 512GB NVMe

    • The caveats

    There are a few things that I cant seem to make work straight out of the box, and that is the SD Card reader, the backlight, and the audio is a bit finicky. Also the trackpad doesn’t respond to two finger scrolling. The wiki is mostly up to date, there are a few edits that need to be made still but there is a bug where I cant register an account yet so I haven’t made all the changes.

    • Processor

    It works like any other Intel processor. Pstates and throttling work.

    • Graphics

    The boot menu sets itself to what looks like 1024x768, but works as you expect in a tiny window. The text console does the full 3200x1800 resolution, but the text is ultra tiny. There isnt a font for the console that covers hidpi screens yet. As for X Windows it requres the drm-kmod-next package. Once installed follow the directions from the package and it works with almost no fuss. I have it running on X with full intel acceleration, but it is running at it’s full 3200x1800 resolution, to scale that down just do xrandr --output eDP-1 --scale 0.5x0.5 it will blow it up to roughly 200%. Due to limitations with X windows and hidpi it is harder to get more granular.

    • Intel Wireless 8265

    The wireless uses the iwm module, as of right now it does not seem to automagically load right now. Adding iwm_load=“YES” will cause the module to load on boot and kldload iwm

    • Battery

    I seem to be getting about 5 hours out of the battery, but everything reports out of the box as expected. I could get more by throttling the CPU down speed wise.

    • Overall impression

    It is a pretty decent experience. While not as polished as a Thinkpad there is a lot of potential with a bit of work and polishing. The laptop itself is not bad, the keyboard is responsive. The build quality is pretty solid. My only real complaint is the trackpad is stiff to click and sort of tiny. They seem to be a bit indifferent to non linux OSes running on the gear but that isnt anything new. I wont have any problems using it and is enough that when I work through this laptop, but I’m not sure at this stage if my next machine will be a System76 laptop, but they have impressed me enough to put them in the running when I go to look for my next portable machine but it hasn’t yet replaced the hole left in my heart by lenovo messing with the thinkpad.

    ###Hardware accelerated AES/HMAC-SHA on octeons

    In this commit, visa@ submitted code (disabled for now) to use built-in acceleration on octeon CPUs, much like AESNI for x86s. I decided to test tcpbench(1) and IPsec, before and after updating and enabling the octcrypto(4) driver. I didn't capture detailed perf stats from before the update, I had heard someone say that Edgerouter Lite boxes would only do some 6MBit/s over ipsec, so I set up a really simple ipsec.conf with ike esp from A to B leading to a policy of esp tunnel from A to B spi 0xdeadbeef auth hmac-sha2-256 enc aes going from one ERL to another (I collect octeons, so I have a bunch to test with) and let tcpbench run for a while on it. My numbers hovered around 7Mbit/s, which coincided with what I've heard, and also that most of the CPU gets used while doing it. Then I edited /sys/arch/octeon/conf/GENERIC, removed the # from octcrypto0 at mainbus0 and recompiled. Booted into the new kernel and got a octcrypto0 line in dmesg, and it was time to rock the ipsec tunnel again. The crypto algorithm and HMAC used by default on ipsec coincides nicely with the list of accelerated functions provided by the driver. Before we get to tunnel traffic numbers, just one quick look at what systat pigs says while the ipsec is running at full steam: PID USER NAME CPU 20\ 40\ 60\ 80\ 100\ 58917 root crypto 52.25 ################# 42636 root softnet 42.48 ############## (idle) 29.74 ######### 1059 root tcpbench 24.22 ####### 67777 root crynlk 19.58 ###### So this indicates that the load from doing ipsec and generating the traffic is somewhat nicely evened out over the two cores in the Edgerouter, and there's even some CPU left unused, which means I can actually ssh into it and have it usable. I have had it running for almost 2 days now, moving some 2.1TB over the tunnel. Now for the new and improved performance numbers: 204452123 4740752 37.402 100.00% Conn: 1 Mbps: 37.402 Peak Mbps: 58.870 Avg Mbps: 37.402 204453149 4692968 36.628 100.00% Conn: 1 Mbps: 36.628 Peak Mbps: 58.870 Avg Mbps: 36.628 204454167 5405552 42.480 100.00% Conn: 1 Mbps: 42.480 Peak Mbps: 58.870 Avg Mbps: 42.480 204455188 5202496 40.804 100.00% Conn: 1 Mbps: 40.804 Peak Mbps: 58.870 Avg Mbps: 40.804 204456194 5062208 40.256 100.00% Conn: 1 Mbps: 40.256 Peak Mbps: 58.870 Avg Mbps: 40.256 The tcpbench numbers fluctuate up and down a bit, but the output is nice enough to actually keep tabs on the peak values. Peaking to 58.8MBit/s! Of course, as you can see, the average is lower but nice anyhow. A manyfold increase in performance, which is good enough in itself, but also moves the throughput from a speed that would make a poor but cheap gateway to something actually useful and decent for many home network speeds. Biggest problem after this gets enabled will be that my options to buy cheap used ERLs diminish.

    ##Beastie Bits

    Tarsnap

    ##Feedback/Questions

    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Thu, 14 Jun 2018 04:00:00 -0700
    Episode 249: Router On A Stick | BSD Now 249
    OpenZFS and DTrace updates in NetBSD, NetBSD network security stack audit, Performance of MySQL on ZFS, OpenSMTP results from p2k18, legacy Windows backup to FreeNAS, ZFS block size importance, and NetBSD as router on a stick.

    ##Headlines
    ZFS and DTrace update lands in NetBSD

    merge a new version of the CDDL dtrace and ZFS code. This changes the upstream vendor from OpenSolaris to FreeBSD, and this version is based on FreeBSD svn r315983.

    • r315983 is from March 2017 (14 months ago), so there is still more work to do

    in addition to the 10 years of improvements from upstream, this version also has these NetBSD-specific enhancements:

    • dtrace FBT probes can now be placed in kernel modules.
    • ZFS now supports mmap().
    • This brings NetBSD 10 years forward, and they should be able to catch the rest of the way up fairly quickly

    ###NetBSD network stack security audit

    • Maxime Villard has been working on an audit of the NetBSD network stack, a project sponsored by The NetBSD Foundation, which has served all users of BSD-derived operating systems.

    Over the last five months, hundreds of patches were committed to the source tree as a result of this work. Dozens of bugs were fixed, among which a good number of actual, remotely-triggerable vulnerabilities.

    Changes were made to strengthen the networking subsystems and improve code quality: reinforce the mbuf API, add many KASSERTs to enforce assumptions, simplify packet handling, and verify compliance with RFCs. This was done in several layers of the NetBSD kernel, from device drivers to L4 handlers.
    In the course of investigating several bugs discovered in NetBSD, I happened to look at the network stacks of other operating systems, to see whether they had already fixed the issues, and if so how. Needless to say, I found bugs there too.

    • A lot of code is shared between the BSDs, so it is especially helpful when one finds a bug, to check the other BSDs and share the fix.

    The IPv6 Buffer Overflow: The overflow allowed an attacker to write one byte of packet-controlled data into ‘packetstorage+off’, where ‘off’ could be approximately controlled too. This allowed at least a pretty bad remote DoS/Crash
    The IPsec Infinite Loop: When receiving an IPv6-AH packet, the IPsec entry point was not correctly computing the length of the IPv6 suboptions, and this, before authentication. As a result, a specially-crafted IPv6 packet could trigger an infinite loop in the kernel (making it unresponsive). In addition this flaw allowed a limited buffer overflow - where the data being written was however not controllable by the attacker.
    The IPPROTO Typo: While looking at the IPv6 Multicast code, I stumbled across a pretty simple yet pretty bad mistake: at one point the Pim6 entry point would return IPPROTONONE instead of IPPROTODONE. Returning IPPROTONONE was entirely wrong: it caused the kernel to keep iterating on the IPv6 packet chain, while the packet storage was already freed.
    The PF Signedness Bug: A bug was found in NetBSD’s implementation of the PF firewall, that did not affect the other BSDs. In the initial PF code a particular macro was used as an alias to a number. This macro formed a signed integer. NetBSD replaced the macro with a sizeof(), which returns an unsigned result.
    The NPF Integer Overflow: An integer overflow could be triggered in NPF, when parsing an IPv6 packet with large options. This could cause NPF to look for the L4 payload at the wrong offset within the packet, and it allowed an attacker to bypass any L4 filtering rule on IPv6.
    The IPsec Fragment Attack: I noticed some time ago that when reassembling fragments (in either IPv4 or IPv6), the kernel was not removing the MPKTHDR flag on the secondary mbufs in mbuf chains. This flag is supposed to indicate that a given mbuf is the head of the chain it forms; having the flag on secondary mbufs was suspicious.
    What Now: Not all protocols and layers of the network stack were verified, because of time constraints, and also because of unexpected events: the recent x86 CPU bugs, which I was the only one able to fix promptly. A todo list will be left when the project end date is reached, for someone else to pick up. Me perhaps, later this year? We’ll see.
    This security audit of NetBSD’s network stack is sponsored by The NetBSD Foundation, and serves all users of BSD-derived operating systems. The NetBSD Foundation is a non-profit organization, and welcomes any donations that help continue funding projects of this kind.

    DigitalOcean

    ###MySQL on ZFS Performance

    I used sysbench to create a table of 10M rows and then, using export/import tablespace, I copied it 329 times. I ended up with 330 tables for a total size of about 850GB. The dataset generated by sysbench is not very compressible, so I used lz4 compression in ZFS. For the other ZFS settings, I used what can be found in my earlier ZFS posts but with the ARC size limited to 1GB. I then used that plain configuration for the first benchmarks. Here are the results with the sysbench point-select benchmark, a uniform distribution and eight threads. The InnoDB buffer pool was set to 2.5GB.
    In both cases, the load is IO bound. The disk is doing exactly the allowed 3000 IOPS. The above graph appears to be a clear demonstration that XFS is much faster than ZFS, right? But is that really the case? The way the dataset has been created is extremely favorable to XFS since there is absolutely no file fragmentation. Once you have all the files opened, a read IOP is just a single fseek call to an offset and ZFS doesn’t need to access any intermediate inode. The above result is about as fair as saying MyISAM is faster than InnoDB based only on table scan performance results of unfragmented tables and default configuration. ZFS is much less affected by the file level fragmentation, especially for point access type.

    ZFS stores the files in B-trees in a very similar fashion as InnoDB stores data. To access a piece of data in a B-tree, you need to access the top level page (often called root node) and then one block per level down to a leaf-node containing the data. With no cache, to read something from a three levels B-tree thus requires 3 IOPS.

    The extra IOPS performed by ZFS are needed to access those internal blocks in the B-trees of the files. These internal blocks are labeled as metadata. Essentially, in the above benchmark, the ARC is too small to contain all the internal blocks of the table files’ B-trees. If we continue the comparison with InnoDB, it would be like running with a buffer pool too small to contain the non-leaf pages. The test dataset I used has about 600MB of non-leaf pages, about 0.1% of the total size, which was well cached by the 3GB buffer pool. So only one InnoDB page, a leaf page, needed to be read per point-select statement.

    To correctly set the ARC size to cache the metadata, you have two choices. First, you can guess values for the ARC size and experiment. Second, you can try to evaluate it by looking at the ZFS internal data. Let’s review these two approaches.

    You’ll read/hear often the ratio 1GB of ARC for 1TB of data, which is about the same 0.1% ratio as for InnoDB. I wrote about that ratio a few times, having nothing better to propose. Actually, I found it depends a lot on the recordsize used. The 0.1% ratio implies a ZFS recordsize of 128KB. A ZFS filesystem with a recordsize of 128KB will use much less metadata than another one using a recordsize of 16KB because it has 8x fewer leaf pages. Fewer leaf pages require less B-tree internal nodes, hence less metadata. A filesystem with a recordsize of 128KB is excellent for sequential access as it maximizes compression and reduces the IOPS but it is poor for small random access operations like the ones MySQL/InnoDB does.

    • In order to improve ZFS performance, I had 3 options:
    • Increase the ARC size to 7GB
    • Use a larger Innodb page size like 64KB
    • Add a L2ARC

    I was reluctant to grow the ARC to 7GB, which was nearly half the overall system memory. At best, the ZFS performance would only match XFS. A larger InnoDB page size would increase the CPU load for decompression on an instance with only two vCPUs; not great either. The last option, the L2ARC, was the most promising.

    ZFS is much more complex than XFS and EXT4 but, that also means it has more tunables/options. I used a simplistic setup and an unfair benchmark which initially led to poor ZFS results. With the same benchmark, very favorable to XFS, I added a ZFS L2ARC and that completely reversed the situation, more than tripling the ZFS results, now 66% above XFS.

    • Conclusion

    We have seen in this post why the general perception is that ZFS under-performs compared to XFS or EXT4. The presence of B-trees for the files has a big impact on the amount of metadata ZFS needs to handle, especially when the recordsize is small. The metadata consists mostly of the non-leaf pages (or internal nodes) of the B-trees. When properly cached, the performance of ZFS is excellent. ZFS allows you to optimize the use of EBS volumes, both in term of IOPS and size when the instance has fast ephemeral storage devices. Using the ephemeral device of an i3.large instance for the ZFS L2ARC, ZFS outperformed XFS by 66%.

    ###OpenSMTPD new config

    TL;DR: OpenBSD #p2k18 hackathon took place at Epitech in Nantes. I was organizing the hackathon but managed to make progress on OpenSMTPD. As mentioned at EuroBSDCon the one-line per rule config format was a design error. A new configuration grammar is almost ready and the underlying structures are simplified. Refactor removes ~750 lines of code and solves _many issues that were side-effects of the design error. New features are going to be unlocked thanks to this.
    • Anatomy of a design error

    OpenSMTPD started ten years ago out of dissatisfaction with other solutions, mainly because I considered them way too complex for me not to get things wrong from time to time.
    The initial configuration format was very different, I was inspired by pyr@’s hoststated, which eventually became relayd, and designed my configuration format with blocks enclosed by brackets.
    When I first showed OpenSMTPD to pyr@, he convinced me that PF-like one-line rules would be awesome, and it was awesome indeed.
    It helped us maintain our goal of simple configuration files, it helped fight feature creeping, it helped us gain popularity and become a relevant MTA, it helped us get where we are now 10 years later.
    That being said, I believe this was a design error. A design error that could not have been predicted until we hit the wall to understand WHY this was an error. One-line rules are semantically wrong, they are SMTP wrong, they are wrong.
    One-line rules are making the entire daemon more complex, preventing some features from being implemented, making others more complex than they should be, they no longer serve our goals.
    To get to the point: we should move to two-line rules :-)

    Anatomy of a design error
    OpenSMTPD started ten years ago out of dissatisfaction with other solutions, mainly because I considered them way too complex for me not to get things wrong from time to time.

    The initial configuration format was very different, I was inspired by pyr@’s hoststated, which eventually became relayd, and designed my configuration format with blocks enclosed by brackets.

    When I first showed OpenSMTPD to pyr@, he convinced me that PF-like one-line rules would be awesome, and it was awesome indeed.

    It helped us maintain our goal of simple configuration files, it helped fight feature creeping, it helped us gain popularity and become a relevant MTA, it helped us get where we are now 10 years later.

    That being said, I believe this was a design error. A design error that could not have been predicted until we hit the wall to understand WHY this was an error. One-line rules are semantically wrong, they are SMTP wrong, they are wrong.

    One-line rules are making the entire daemon more complex, preventing some features from being implemented, making others more complex than they should be, they no longer serve our goals.

    To get to the point: we should move to two-line rules :-)

    • The problem with one-line rules

    OpenSMTPD decides to accept or reject messages based on one-line rules such as:

    accept from any for domain poolp.org deliver to mbox

    Which can essentially be split into three units:

    • the decision: accept/reject
    • the matching: from any for domain poolp.org
    • the (default) action: deliver to mbox

    To ensure that we meet the requirements of the transactions, the matching must be performed during the SMTP transaction before we take a decision for the recipient.
    Given that the rule is atomic, that it doesn’t have an identifier and that the action is part of it, the two only ways to make sure we can remember the action to take later on at delivery time is to either:

    • save the action in the envelope, which is what we do today
    • evaluate the envelope again at delivery
    • And this this where it gets tricky… both solutions are NOT ok.

    The first solution, which we’ve been using for a decade, was to save the action within the envelope and kind of carve it in stone. This works fine… however it comes with the downsides that errors fixed in configuration files can’t be caught up by envelopes, that delivery action must be validated way ahead of time during the SMTP transaction which is much trickier, that the parsing of delivery methods takes place as the _smtpd user rather than the recipient user, and that envelope structures that are passed all over OpenSMTPD carry delivery-time informations, and more, and more, and more. The code becomes more complex in general, less safe in some particular places, and some areas are nightmarish to deal with because they have to deal with completely unrelated code that can’t be dealt with later in the code path.

    The second solution can’t be done. An envelope may be the result of nested rules, for example an external client, hitting an alias, hitting a user with a .forward file resolving to a user. An envelope on disk may no longer match any rule or it may match a completely different rule If we could ensure that it matched the same rule, evaluating the ruleset may spawn new envelopes which would violate the transaction. Trying to imagine how we could work around this leads to more and more and more RFC violations, incoherent states, duplicate mails, etc…

    There is simply no way to deal with this with atomic rules, the matching and the action must be two separate units that are evaluated at two different times, failure to do so will necessarily imply that you’re either using our first solution and all its downsides, or that you are currently in a world of pain trying to figure out why everything is burning around you. The minute the action is written to an on-disk envelope, you have failed.

    A proper ruleset must define a set of matching patterns resolving to an action identifier that is carved in stone, AND a set of named action set that is resolved dynamically at delivery time.

    • Follow the link above to see the rest of the article

    Break

    ##News Roundup
    Backing up a legacy Windows machine to a FreeNAS with rsync

    I have some old Windows servers (10 years and counting) and I have been using rsync to back them up to my FreeNAS box. It has been working great for me.

    First of all, I do have my Windows servers backup in virtualized format. However, those are only one-time snapshops that I run once in a while. These are classic ASP IIS web servers that I can easily put up on a new VM. However, many of these legacy servers generate gigabytes of data a day in their repositories. Running VM conversion daily is not ideal.

    My solution was to use some sort of rsync solution just for the data repos. I’ve tried some applications that didn’t work too well with Samba shares and these old servers have slow I/O. Copying files to external sata or usb drive was not ideal. We’ve moved on from Windows to Linux and do not have any Windows file servers of capacity to provide network backups. Hence, I decided to use Delta Copy with FreeNAS. So here is a little write up on how to set it up. I have 4 Windows 2000 servers backing up daily with this method.

    First, download Delta Copy and install it. It is open-source and pretty much free. It is basically a wrapper for cygwin’s rsync. When you install it, it will ask you to install the Server services which allows you to run it as a Rsync server on Windows. You don’t need to do this. Instead, you will be just using the Delta Copy Client application. But before we do that, we will need to configure our Rsync service for our Windows Clients on FreeNAS.

    • In FreeNAS, go under Services , Select Rsync > Rsync Modules > Add Rsync Module.
    • Then fill out the form; giving the module a name and set the path. In my example, I simply called it WIN and linked it to a user called backupuser.
    • This process is much easier than trying to configure the daemon rsyncd.conf file by hand.
    • Now, on the Windows Client, start the DeltaCopy Client. You will create a new Profile.
    • You will need to enter the IP of the Rsync server (FreeNAS) and specify the module name which will be called “Virtual Directory Name.” When you pull the select menu, the list of Rsync Modules you created earlier in FreeNAS will populate.
    • You can set authentication. On the server, you can restrict by IP and do other things to lock down your rsync.
    • Next, you will add folders (and/or files) you want to synchronize.
    • Once the paths are set up, you can run a sync by right clicking the profile name.
    • Here, I made a test sync to a home folder of a virtualized windows box. As you can see, I mounted the rsync volume on my mac to see the progress. The rsync worked beautifully. DeltaCopy did what it was told.
    • Once you get everything working. The next thing to do is set schedules. If you done tasks schedules in Windows before, it is pretty straightforward. DeltaCopy has a link in the application to directly create a new task for you. I set my backups to run nightly and it has been working great.

    There you have it. Windows rsync to FreeNAS using DeltaCopy.
    The nice thing about FreeNAS is you don’t have to modify /etc/rsyncd.conf files. Everything can be done in the web admin.

    iXsystems

    ###How to write ATF tests for NetBSD

    I have recently started contributing to the amazing NetBSD foundation. I was thinking of trying out a new OS for a long time. Switching to the NetBSD OS has been a fun change.

    My first contribution to the NetBSD foundation was adding regression tests for the Address Sanitizer (ASan) in the Automated Testing Framework(ATF) which NetBSD has. I managed to complete it with the help of my really amazing mentor Kamil. This post is gonna be about the ATF framework that NetBSD has and how to you can add multiple tests with ease.

    • Intro

    In ATF tests we will basically be talking about test programs which are a suite of test cases for a specific application or program.

    • The ATF suite of Commands

    There are a variety of commands that the atf suite offers. These include :

    • atf-check: The versatile command that is a vital part of the checking process. man page

    • atf-run: Command used to run a test program. man page

    • atf-fail: Report failure of a test case.

    • atf-report: used to pretty print the atf-run. man page

    • atf-set: To set atf test conditions.

    • We will be taking a better look at the syntax and usage later.

    • Let’s start with the Basics

    The ATF testing framework comes preinstalled with a default NetBSD installation. It is used to write tests for various applications and commands in NetBSD. One can write the Test programs in either the C language or in shell script. In this post I will be dealing with the Bash part.

    • Follow the link above to see the rest of the article

    ###The Importance of ZFS Block Size

    • Warning! WARNING! Don’t just do things because some random blog says so

    One of the important tunables in ZFS is the recordsize (for normal datasets) and volblocksize (for zvols). These default to 128KB and 8KB respectively.
    As I understand it, this is the unit of work in ZFS. If you modify one byte in a large file with the default 128KB record size, it causes the whole 128KB to be read in, one byte to be changed, and a new 128KB block to be written out.
    As a result, the official recommendation is to use a block size which aligns with the underlying workload: so for example if you are using a database which reads and writes 16KB chunks then you should use a 16KB block size, and if you are running VMs containing an ext4 filesystem, which uses a 4KB block size, you should set a 4KB block size
    You can see it has a 16GB total file size, of which 8.5G has been touched and consumes space - that is, it’s a “sparse” file. The used space is also visible by looking at the zfs filesystem which this file resides in
    Then I tried to copy the image file whilst maintaining its “sparseness”, that is, only touching the blocks of the zvol which needed to be touched. The original used only 8.42G, but the copy uses 14.6GB - almost the entire 16GB has been touched! What’s gone wrong?
    I finally realised that the difference between the zfs filesystem and the zvol is the block size. I recreated the zvol with a 128K block size
    That’s better. The disk usage of the zvol is now exactly the same as for the sparse file in the filesystem dataset

    • It does impact the read speed too. 4K blocks took 5:52, and 128K blocks took 3:20
    • Part of this is the amount of metadata that has to be read, see the MySQL benchmarks from earlier in the show
    • And yes, using a larger block size will increase the compression efficiency, since the compressor has more redundant data to optimize.
    • Some of the savings, and the speedup is because a lot less metadata had to be written
    • Your zpool layout also plays a big role, if you use 4Kn disks, and RAID-Z2, using a volblocksize of 8k will actually result in a large amount of wasted space because of RAID-Z padding. Although, if you enable compression, your 8k records may compress to only 4k, and then all the numbers change again.

    ###Using a Raspberry Pi 2 as a Router on a Stick Starring NetBSD

    • Sorry we didn’t answer you quickly enough

    A few weeks ago I set about upgrading my feeble networking skills by playing around with a Cisco 2970 switch. I set up a couple of VLANs and found the urge to set up a router to route between them. The 2970 isn’t a modern layer 3 switch so what am I to do?

    Why not make use of the Raspberry Pi 2 that I’ve never used and put it to some good use as a ‘router on a stick’.

    I could install a Linux based OS as I am quite familiar with it but where’s the fun in that? In my home lab I use SmartOS which by the way is a shit hot hypervisor but as far as I know there aren’t any Illumos distributions for the Raspberry Pi. On the desktop I use Solus OS which is by far the slickest Linux based OS that I’ve had the pleasure to use but Solus’ focus is purely desktop. It’s looking like BSD then!

    I believe FreeBSD is renowned for it’s top notch networking stack and so I wrote to the BSDNow show on Jupiter Broadcasting for some help but it seems that the FreeBSD chaps from the show are off on a jolly to some BSD conference or another(love the show by the way).

    It looks like me and the luvverly NetBSD are on a date this Saturday. I’ve always had a secret love for NetBSD. She’s a beautiful, charming and promiscuous lover(looking at the supported architectures) and I just can’t stop going back to her despite her misgivings(ahem, zfs). Just my type of grrrl!

    Let’s crack on…

    • Follow the link above to see the rest of the article

    ##Beastie Bits

    Tarsnap

    ##Feedback/Questions

    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Wed, 06 Jun 2018 11:00:00 -0700
    Episode 248: Show Me The Mooney | BSD Now 248
    DragonflyBSD release 5.2.1 is here, BPF kernel exploit writeup, Remote Debugging the running OpenBSD kernel, interview with Patrick Mooney, FreeBSD buildbot setup in a jail, dumping your USB, and 5 years of gaming on FreeBSD. Headlines DragonFlyBSD: release52 (w/stable HAMMER2, as default root)
    • DragonflyBSD 5.2.1 was released on May 21, 2018
    • > Big Ticket items: Meltdown and Spectre mitigation support Meltdown isolation and spectre mitigation support added. Meltdown mitigation is automatically enabled for all Intel cpus. Spectre mitigation must be enabled manually via sysctl if desired, using sysctls machdep.spectremitigation and machdep.meltdownmitigation. HAMMER2 H2 has received a very large number of bug fixes and performance improvements. We can now recommend H2 as the default root filesystem in non-clustered mode. Clustered support is not yet available. ipfw Updates Implement state based "redirect", i.e. without using libalias. ipfw now supports all possible ICMP types. Fix ICMPMAXTYPE assumptions (now 40 as of this release). Improved graphics support The drm/i915 kernel driver has been updated to support Intel Coffeelake GPUs Add 24-bit pixel format support to the EFI frame buffer code. Significantly improve fbio support for the "scfb" XOrg driver. This allows EFI frame buffers to be used by X in situations where we do not otherwise support the GPU. Partly implement the FBIOBLANK ioctl for display powersaving. Syscons waits for drm modesetting at appropriate places, avoiding races.
    PS4 4.55 BPF Race Condition Kernel Exploit Writeup

    Note: While this bug is primarily interesting for exploitation on the PS4, this bug can also potentially be exploited on other unpatched platforms using FreeBSD if the attacker has read/write permissions on /dev/bpf, or if they want to escalate from root user to kernel code execution. As such, I've published it under the "FreeBSD" folder and not the "PS4" folder.

    • Introduction

    Welcome to the kernel portion of the PS4 4.55FW full exploit chain write-up. This bug was found by qwerty, and is fairly unique in the way it's exploited, so I wanted to do a detailed write-up on how it worked. The full source of the exploit can be found here. I've previously covered the webkit exploit implementation for userland access here.

    • FreeBSD or Sony's fault? Why not both...

    Interestingly, this bug is actually a FreeBSD bug and was not (at least directly) introduced by Sony code. While this is a FreeBSD bug however, it's not very useful for most systems because the /dev/bpf device driver is root-owned, and the permissions for it are set to 0600 (meaning owner has read/write privileges, and nobody else does) - though it can be used for escalating from root to kernel mode code execution. However, let’s take a look at the make_dev() call inside the PS4 kernel for /dev/bpf (taken from a 4.05 kernel dump).

    seg000:FFFFFFFFA181F15B lea rdi, unk_FFFFFFFFA2D77640 seg000:FFFFFFFFA181F162 lea r9, aBpf ; "bpf" seg000:FFFFFFFFA181F169 mov esi, 0 seg000:FFFFFFFFA181F16E mov edx, 0 seg000:FFFFFFFFA181F173 xor ecx, ecx seg000:FFFFFFFFA181F175 mov r8d, 1B6h seg000:FFFFFFFFA181F17B xor eax, eax seg000:FFFFFFFFA181F17D mov cs:qword_FFFFFFFFA34EC770, 0 seg000:FFFFFFFFA181F188 call make_dev

    We see UID 0 (the UID for the root user) getting moved into the register for the 3rd argument, which is the owner argument. However, the permissions bits are being set to 0x1B6, which in octal is 0666. This means anyone can open /dev/bpf with read/write privileges. I’m not sure why this is the case, qwerty speculates that perhaps bpf is used for LAN gaming. In any case, this was a poor design decision because bpf is usually considered privileged, and should not be accessible to a process that is completely untrusted, such as WebKit. On most platforms, permissions for /dev/bpf will be set to 0x180, or 0600.

    • Race Conditions - What are they?

    The class of the bug abused in this exploit is known as a "race condition". Before we get into bug specifics, it's important for the reader to understand what race conditions are and how they can be an issue (especially in something like a kernel). Often in complex software (such as a kernel), resources will be shared (or "global"). This means other threads could potentially execute code that will access some resource that could be accessed by another thread at the same point in time. What happens if one thread accesses this resource while another thread does without exclusive access? Race conditions are introduced.

    Race conditions are defined as possible scenarios where events happen in a sequence different than the developer intended which leads to undefined behavior. In simple, single-threaded programs, this is not an issue because execution is linear. In more complex programs where code can be running in parallel however, this becomes a real issue. To prevent these problems, atomic instructions and locking mechanisms were introduced. When one thread wants to access a critical resource, it will attempt to acquire a "lock". If another thread is already using this resource, generally the thread attempting to acquire the lock will wait until the other thread is finished with it. Each thread must release the lock to the resource after they're done with it, failure to do so could result in a deadlock.

    While locking mechanisms such as mutexes have been introduced, developers sometimes struggle to use them properly. For example, what if a piece of shared data gets validated and processed, but while the processing of the data is locked, the validation is not? There is a window between validation and locking where that data can change, and while the developer thinks the data has been validated, it could be substituted with something malicious after it is validated, but before it is used. Parallel programming can be difficult, especially when, as a developer, you also want to factor in the fact that you don't want to put too much code in between locking and unlocking as it can impact performance.

    iXsystems

    Remote Debugging the running OpenBSD kernel
    • Subtitled: A way to understand the OpenBSD internals +> The Problem +> A few month ago, I tried porting the FreeBSD kdb along with it's gdb stub implementations to OpenBSD as a practice of learning the internals of an BSD operating system. The ddb code in both FreeBSD and OpenBSD looks pretty much the same and the GDB Remote Serial Protocol looks very minimal. +> But sadly I got very busy and the work is stalled but I'm planning on resuming the attempt as soon as I get the chance, But there is an alternative way to Debugging the OpenBSD kernel via QEMU. What I did below is basically the same with a few minor changes which I hope to describe it as best. +> Installing OpenBSD on Qemu +> For debugging the kernel, we need a working OpenBSD system running on Qemu. I chose to create a raw disk file to be able to easily mount it later via the host and copy the custom kernel onto it. $ qemu-img create -f raw disk.raw 5G $ qemu-system-x8664 -m 256M \ -drive format=raw,file=install63.fs \ -drive format=raw,file=disk.raw +> Custom Kernel +> To debug the kernel, we need a version of the kernel with debugging symbols and for that we have to recompile it first. The process is documented at Building the System from Source: ... +> Then we can copy the bsd kernel to the guest machine and keep the bsd.gdb on the host to start the remote debugging via gdb. +> Remote debugging kernel +> Now it's to time to boot the guest with the new custom kernel. Remember that the -s argument enables the gdb server on qemu on localhost port 1234 by default: $ qemu-system-x8664 -m 256M -s \ -net nic -net user \ -drive format=raw,file=install63.fs \ +> Now to finally attach to the running kernel:
    Interview - Patrick Mooney - Software Engineer pmooney@pfmooney.com / @pfmooney
    • BR: How did you first get introduced to UNIX?
    • AJ: What got you started contributing to an open source project?
    • BR: What sorts of things have you worked on in the past?
    • AJ: Can you tell us more about what attracted you to illumos?
    • BR: How did you get interested in, and started with, systems development?
    • AJ: When did you first get interested in bhyve?
    • BR: How much work was it to take the years-old port of bhyve and get it working on modern IllumOS?
    • AJ: What was the process for getting the bhyve port caught up to current FreeBSD?
    • BR: How usable is bhyve on illumOS?
    • AJ: What area are you most interested in improving in bhyve?
    • BR: Do you think the FreeBSD and illumos versions of bhyve will stay in sync with each other?
    • AJ: What do you do for fun?
    • BR: Anything else you want to mention?
    News Roundup Setting up buildbot in FreeBSD Jails

    In this article, I would like to present a tutorial to set up buildbot, a continuous integration (CI) software (like Jenkins, drone, etc.), making use of FreeBSD’s containerization mechanism "jails". We will cover terminology, rationale for using both buildbot and jails together, and installation steps. At the end, you will have a working buildbot instance using its sample build configuration, ready to play around with your own CI plans (or even CD, it’s very flexible!). Some hints for production-grade installations are given, but the tutorial steps are meant for a test environment (namely a virtual machine). Buildbot’s configuration and detailed concepts are not in scope here.

    • Table of contents

      • Choosing host operating system and version for buildbot
      • Create a FreeBSD playground
      • Introduction to jails
      • Overview of buildbot
      • Set up jails
      • Install buildbot master
      • Run buildbot master
      • Install buildbot worker
      • Run buildbot worker
      • Set up web server nginx to access buildbot UI
      • Run your first build
      • Production hints
      • Finished!
    • Choosing host operating system and version for buildbot

    We choose the released version of FreeBSD (11.1-RELEASE at the moment). There is no particular reason for it, and as a matter of fact buildbot as a Python-based server is very cross-platform; therefore the underlying OS platform and version should not make a large difference.

    It will make a difference for what you do with buildbot, however. For instance, poudriere is the de-facto standard for building packages from source on FreeBSD. Builds run in jails which may be any FreeBSD base system version older or equal to the host’s version (reason will be explained below). In other words, if the host is FreeBSD 11.1, build jails created by poudriere could e.g. use 9.1, 10.3, 11.0, 11.1, but potentially not version 12 or newer because of incompatibilities with the host’s kernel (jails do not run their own kernel as full virtual machines do). To not prolong this article over the intended scope, the details of which nice things could be done or automated with buildbot are not covered.

    Package names on the FreeBSD platform are independent of the OS version, since external software (as in: not part of base system) is maintained in FreeBSD ports. So, if your chosen FreeBSD version (here: 11) is still officially supported, the packages mentioned in this post should work. In the unlikely event of package name changes before you read this article, you should be able to find the actual package names like pkg search buildbot.

    Other operating systems like the various Linux distributions will use different package names but might also offer buildbot pre-packaged. If not, the buildbot installation manual offers steps to install it manually. In such case, the downside is that you will have to maintain and update the buildbot modules outside the stability and (semi-)automatic updates of your OS packages.

    DigitalOcean

    Dumping your USB

    One of the many new features of OpenBSD 6.3 is the possibility to dump USB traffic to userland via bpf(4). This can be done with tcpdump(8) by specifying a USB bus as interface:

    ```

    tcpdump -Xx -i usb0

    tcpdump: listening on usb0, link-type USBPCAP 12:28:03.317945 bus 0 < addr 1: ep1 intr 2 0000: 0400 ..

    12:28:03.318018 bus 0 > addr 1: ep0 ctrl 8 0000: 00a3 0000 0002 0004 00 .........
    [...] ```

    As you might have noted I decided to implement the existing USBPcap capture format. A capture format is required because USB packets do not include all the necessary information to properly interpret them. I first thought I would implement libpcap's DLTUSB but then I quickly realize that this was not a standard. It is instead a FreeBSD specific format which has been since then renamed DLTUSBFREEBSD. But I didn't want to embrace xkcd #927, so I look at the existing formats: DLTUSBFREEBSD, DLTUSBLINUX, DLTUSBLINUXMMAPPED, DLTUSBDARWIN and DLT_USBPCAP. I was first a bit sad to see that nobody could agree on a common format then I moved on and picked the simplest one: USBPcap. Implementing an already existing format gives us out-of-box support for all the tools supporting it. That's why having common formats let us share our energy. In the case of USBPcap it is already supported by Wireshark, so you can already inspect your packet graphically. For that you need to first capture raw packets:

    ```

    tcpdump -s 3303 -w usb.pcap -i usb0

    tcpdump: listening on usb0, link-type USBPCAP ^C 208 packets received by filter 0 packets dropped by kernel ```

    USB packets can be quite big, that's why I'm not using tcpdump(8)'s default packet size. In this case, I want to make sure I can dump the complete uaudio(4) frames. It is important to say that what is dumped to userland is what the USB stack sees. Packets sent on the wire might differ, especially when it comes to retries and timing. So this feature is not here to replace any USB analyser, however I hope that it will help people understand how things work and what the USB stack is doing. Even I found some interesting timing issues while implementing isochronous support.

    Run OpenBSD on your web server

    As soon as you're there you can enable an httpd(8) daemon, it's already installed on OpenBSD, you just need to configure it:

    www# vi /etc/httpd.conf

    • Add two server sections---one for www and another for naked domain (all requests are redirected to www).

    ``` server "www.example.com" { listen on * port 80 root "/htdocs/www.example.com" }

    server "example.com" { listen on * port 80 block return 301 "http://www.example.com$REQUEST_URI" } ```

    • httpd is chrooted to /var/www by default, so let's make a document root directory:

    www# mkdir -p /var/www/htdocs/www.example.com

    • Save and check this configuration:

    www# httpd -n configuration ok

    • Enable httpd(8) daemon and start it.

    www# rcctl enable httpd www# rcctl start httpd

    • Publish your website

    • Copy your website content into /var/www/htdocs/www.example.com and then test it your web browser.

    http://XXX.XXX.XXX.XXX/

    Your web server should be up and running.

    • Update DNS records

    If there is another HTTPS server using this domain, configure that server to redirect all HTTPS requests to HTTP.

    Now as your new server is ready you can update DNS records accordingly.

    example.com. 300 IN A XXX.XXX.XXX.XXX www.example.com. 300 IN A XXX.XXX.XXX.XXX

    • Examine your DNS is propagated.

    $ dig example.com www.example.com

    Modern Akonadi and KMail on FreeBSD

    For, quite literally a year or more, KMail and Akonadi on FreeBSD have been only marginally useful, at best. KDE4 era KMail was pretty darn good, but everything after that has had a number of FreeBSD users tearing out their hair. Sure, you can go to Trojitá, which has its own special problems and is generally “meh”, or bail out entirely to webmail, but .. KMail is a really great mail client when it works. Which, on Linux desktops, is nearly always, and on FreeBSD, is was nearly never.

    I looked at it with Dan and Volker last summer, briefly, and we got not much further than “hmm”. There’s a message about “The world is going to end!” which hardly makes sense, it means that a message has been truncated or corrupted while traversing a UNIX domain socket.

    Now Alexandre Martins — praise be! — has wandered in with a likely solution. KDE Bug 381850 contains a suggestion, which deserves to be publicised (and tested):

    sysctl net.local.stream.recvspace=65536 sysctl net.local.stream.sendspace=65536

    The default FreeBSD UNIX local socket buffer space is 8kiB. Bumping the size up to 64kiB — which matches the size that Linux has by default — suddenly makes KMail and Akonadi shine again. No other changes, no recompiling, just .. bump the sysctls (perhaps also in /etc/sysctl.conf) and KMail from Area51 hums along all day without ending the world.

    Since changing this value may have other effects, and Akonadi shouldn’t be dependent on a specific buffer size anyway, I’m looking into the Akonadi code (encouraged by Dan) to either automatically size the socket buffers, or to figure out where in the underlying code the assumption about buffer size lives. So for now, sysctl can make KMail users on FreeBSD happy, and later we hope to have things fully automatic (and if that doesn’t pan out, well, pkg-message exists).

    PS. Modern KDE PIM applications — Akonadi, KMail — which live in the deskutils/ category of the official FreeBSD ports were added to the official tree April 10th, so you can get your fix now from the official tree.

    Beastie Bits

    Tarsnap ad

    Feedback/Questions
    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Tue, 29 May 2018 11:30:00 -0700
    Episode 247: Interning for FreeBSD | BSD Now 247
    FreeBSD internship learnings, exciting developments coming to FreeBSD, running FreeNAS on DigitalOcean, Network Manager control for OpenBSD, OpenZFS User Conference Videos are here and batch editing files with ed. Headlines What I learned during my FreeBSD intership

    Hi, my name is Mitchell Horne. I am a computer engineering student at the University of Waterloo, currently in my third year of studies, and fortunate to have been one of the FreeBSD Foundation’s co-op students this past term (January to April). During this time I worked under Ed Maste, in the Foundation’s small Kitchener office, along with another co-op student Arshan Khanifar. My term has now come to an end, and so I’d like to share a little bit about my experience as a newcomer to FreeBSD and open-source development.

    I’ll begin with some quick background — and a small admission of guilt. I have been an open-source user for a large part of my life. When I was a teenager I started playing around with Linux, which opened my eyes to the wider world of free software. Other than some small contributions to GNOME, my experience has been mostly as an end user; however, the value of these projects and the open-source philosophy was not lost on me, and is most of what motivated my interest in this position. Before beginning this term I had no personal experience with any of the BSDs, although I knew of their existence and was extremely excited to receive the position. I knew it would be a great opportunity for growth, but I must confess that my naivety about FreeBSD caused me to make the silent assumption that this would be a form of compromise — a stepping stone that would eventually allow me to work on open-source projects that are somehow “greater” or more “legitimate”. After four months spent immersed in this project I have learned how it operates, witnessed its community, and learned about its history. I am happy to admit that I was completely mistaken. Saying it now seems obvious, but FreeBSD is a project with its own distinct uses, goals, and identity. For many there may exist no greater opportunity than to work on FreeBSD full time, and with what I know now I would have a hard time coming up with a project that is more “legitimate”.

    • What I Liked

    In all cases, the work I submitted this term was reviewed by no less than two people before being committed. The feedback and criticism I received was always both constructive and to the point, and it commented on everything from high-level ideas to small style issues. I appreciate having these thorough reviews in place, since I believe it ultimately encourages people to accept only their best work. It is indicative of the high quality that already exists within every aspect of this project, and this commitment to quality is something that should continue to be honored as a core value. As I’ve discovered in some of my previous work terms, it is all too easy cut corners in the name of a deadline or changing priorities, but the fact that FreeBSD doesn’t need to make these types of compromises is a testament to the power of free software.

    It’s a small thing, but the quality and completeness of the FreeBSD documentation was hugely helpful throughout my term. Everything you might need to know about utilities, library functions, the kernel, and more can be found in a man page; and the handbook is a great resource as both an introduction to the operating system and a reference. I only wish I had taken some time earlier in the term to explore the different documents more thoroughly, as they cover a wide range of interesting and useful topics. The effort people put into writing and maintaining FreeBSD’s documentation is easy to overlook, but its value cannot be overstated.

    • What I Learned

    Although there was a lot I enjoyed, there were certainly many struggles I faced throughout the term, and lessons to be learned from them. I expect that some of issues I faced may be specific to FreeBSD, while others may be common to open-source projects in general. I don’t have enough experience to speculate on which is which, so I will leave this to the reader.

    The first lesson can be summed up simply: you have to advocate for your own work. FreeBSD is made up in large part by volunteer efforts, and in many cases there is more work to go around than people available to do it. A consequence of this is that there will not be anybody there to check up on you. Even in my position where I actually had a direct supervisor, Ed often had his plate full with so many other things that the responsibility to find someone to look at my work fell to me. Admittedly, a couple of smaller changes I worked on got left behind or stuck in review simply because there wasn’t a clear person/place to reach out to.

    I think this is both a barrier of entry to FreeBSD and a mental hurdle that I needed to get over. If there’s a change you want to see included or reviewed, then you may have to be the one to push for it, and there’s nothing wrong with that. Perhaps this process should be easier for newcomers or infrequent contributors (the disconnect between Bugzilla and Phabricator definitely leaves a lot to be desired), but we also have to be aware that this simply isn’t the reality right now. Getting your work looked at may require a little bit more self-motivation, but I’d argue that there are much worse problems a project like FreeBSD could have than this.

    I understand this a lot better now, but it is still something I struggle with. I’m not naturally the type of person who easily connects with others or asks for help, so I see this as an area for future growth rather than simply a struggle I encountered and overcame over the course of this work term. Certainly it is an important skill to understand the value of your own work, and equally important is the ability to communicate that value to others.

    I also learned the importance of starting small. My first week or two on the job mainly involved getting set up and comfortable with the workflow. After this initial stage, I began exploring the project and found myself overwhelmed by its scale. With so many possible areas to investigate, and so much work happening at once, I felt quite lost on where to begin. Many of the potential projects I found were too far beyond my experience level, and most small bugs were picked up and fixed quickly by more experienced contributors before I could even get to them.

    It’s easy to make the mistake that FreeBSD is made up solely of a few rock-star committers that do everything. This is how it appears at face-value, as reading through commits, bug reports, and mailing lists yields a few of the same names over and over. The reality is that just as important are the hundreds of users and infrequent contributors who take the time to submit bug reports, patches, or feedback. Even though there are some people who would fall under the umbrella of a rock-star committer, they didn’t get there overnight. Rather, they have built their skills and knowledge through many years of involvement in FreeBSD and similar projects.

    As a student coming into this project and having high expectations of myself, it was easy to set the bar too high by comparing myself against those big committers, and feel that my work was insignificant, inadequate, and simply too infrequent. In reality, there is no reason I should have felt this way. In a way, this comparison is disrespectful to those who have reached this level, as it took them a long time to get there, and it’s a humbling reminder that any skill worth learning requires time, patience, and dedication. It is easy to focus on an end product and simply wish to be there, but in order to be truly successful one must start small, and find satisfaction in the struggle of learning something new. I take pride in the many small successes I’ve had throughout my term here, and appreciate the fact that my journey into FreeBSD and open-source software is only just beginning.

    • Closing Thoughts

    I would like to close with some brief thank-you’s. First, to everyone at the Foundation for being so helpful, and allowing this position to exist in the first place. I am extremely grateful to have been given this unique opportunity to learn about and give back to the open-source world. I’d also like to thank my office mates; Ed: for being an excellent mentor, who offered an endless wealth of knowledge and willingness to share it. My classmate and fellow intern Arshan: for giving me a sense of camaraderie and the comforting reminder that at many moments he was as lost as I was. Finally, a quick thanks to everyone else I crossed paths with who offered reviews and advice. I appreciate your help and look forward to working with you all further.

    I am walking away from this co-op with a much greater appreciation for this project, and have made it a goal to remain involved in some capacity. I feel that I’ve gained a little bit of a wider perspective on my place in the software world, something I never really got from my previous co-ops. Whether it ends up being just a stepping stone, or the beginning of much larger involvement, I thoroughly enjoyed my time here.

    Recent Developments in FreeBSD

    DigitalOcean Digital Ocean Promo Link for BSD Now Listeners

    Running FreeNAS on a DigitalOcean Droplet
    • Need to backup your FreeNAS offsite? Run a locked down instance in the cloud, and replicate to it
    • The tutorial walks though the steps of converting a fresh FreeBSD based droplet into a FreeNAS
    • Create a droplet, and add a small secondary block-storage device
    • Boot the droplet, login, and download FreeNAS
    • Disable swap, enable ‘foot shooting’ mode in GEOM
    • use dd to write the FreeNAS installer to the boot disk
    • Reboot the droplet, and use the FreeNAS installer to install FreeNAS to the secondary block storage device
    • Now, reimage the droplet with FreeBSD again, to replace the FreeNAS installer
    • Boot, and dd FreeNAS from the secondary block storage device back to the boot disk
    • You can now destroy the secondary block device
    • Now you have a FreeNAS, and can take it from there.
    • Use the FreeNAS replication wizard to configure sending snapshots from your home NAS to your cloud NAS
    • Note: You might consider creating a new block storage device to create a larger pool, that you can more easily grow over time, rather than using the boot device in the droplet as your main pool.
    News Roundup Network Manager Control for OpenBSD (Updated)
    • Generalities
    • I just remind the scope of this small tool:

      • allow you to pre-define several cable or wifi connections
      • let nmctl to connect automatically to the first available one
      • allow you to easily switch from one network connection to an other one
      • create openbox dynamic menus
    • Enhancements in this version

    This is my second development version: 0.2. I've added performed several changes in the code:

    • code style cleanup, to better match the python recommendations
    • adapt the tool to allow to connect to an Open-wifi having blancs in the name. This happens in some hotels
    • implement a loop as work-around concerning the arp table issue.

    The source code is still on the git of Sourceforge.net. You can see the files here

    And you can download the last version here

    • Feedbacks after few months

    I'm using this script on my OpenBSD laptop since about 5 months. In my case, I'm mainly using the openbox menus and the --restart option.

    • The Openbox menus

    The openbox menus are working fine. As explain in my previous blog, I just have to create 2 entries in my openbox's menu.xml file, and all the rest comes automatically from nmctl itself thanks to the --list and --scan options. I've not changed this part of nmctl since it works as expected (for me :-) ).

    • The --restart option

    Because I'm very lazy, and because OpenBSD is very simple to use, I've added the command "nmctl --restart" in the /etc/apm/resume script. Thanks to apmd, this script will be used each time I'm opening the lid of my laptop. In other words, each time I'll opening my laptop, nmctl will search the optimum network connection for me. But I had several issues in this scenario. Most of the problems were linked to the arp table issues. Indeed, in some circumstances, my proxy IP address was associated to the cable interface instead of the wifi interface or vice-versa. As consequence I'm not able to connect to the proxy, thus not able to connect to internet. So the ping to google (final test nmctl perform) is failing. Knowing that anyhow, I'm doing a full arp cleanup, it's not clear for me from where this problem come from. To solve this situation I've implemented a "retry" concept. In other words, before testing an another possible network connection (as listed in my /etc/nmctl.conf file), the script try 3x the current connection's parameters. If you want to reduce or increase this figures, you can do it via the --retry parameter.

    • Results of my expertise with this small tool

    Where ever I'm located, my laptop is now connecting automatically to the wifi / cable connection previously identified for this location. Currently I have 3 places where I have Wifi credentials and 2 offices places where I just have to plug the network cable. Since the /etc/apm/resume scripts is triggered when I open the lid of the laptop, I just have to make sure that I plug the RJ45 before opening the laptop. For the rest, I do not have to type any commands, OpenBSD do all what is needed ;-). I hotels or restaurants, I can just connect to the Open Wifi thanks to the openbox menu created by "nmctl --scan".

    • Next steps

    • Documentation

    The tool is missing lot of documentation. I appreciate OpenBSD for his great documentation, so I have to do the same. I plan to write a README and a man page at first instances. But since my laziness, I will do it as soon as I see some interest for this tool from other persons.

    • Tests

    I now have to travel and see how to see the script react on the different situations. Interested persons are welcome to share with me the outcome of their tests. I'm curious how it work.

    OpenBSD 6.3 on EdgeRouter Lite simple upgrade method
    • TL;DR

    OpenBSD 6.3 oceton upgrade instructions may not factor that your ERL is running from the USB key they want wiped with the miniroot63.fs image loaded on. Place the bsd.rd for OpenBSD 6.3 on the sd0i slice used by U-Boot for the kernel, and then edit the boot command to run it.

    • a tiny upgrade

    The OpenBSD documentation is comprehensive, but there might be rough corners around what are probably edge cases in their user base. People running EdgeRouter Lite hardware for example, who are looking to upgrade from 6.2 to 6.3. The documentation, which gave us everything we needed last time, left me with some questions about how to upgrade. In INSTALL.octeon, the Upgrading section does mention: The best solution, whenever possible, is to backup your data and reinstall from scratch I had to check if that directive existed in the documentation for other architectures. I wondered if oceton users were getting singled out. We were not. Just simplicity and pragmatism.

    • Reading on:

    To upgrade OpenBSD 6.3 from a previous version, start with the general instructions in the section "Installing OpenBSD". But that section requires us to boot off of TFTP or NFS. Which I don’t want to do right now. Could also use a USB stick with the miniroot63.fs installed on it. But as the ERL only has a single USB port, we would have to remove the USB stick with the current install on it. Once we get to the Install or Upgrade prompt, there would be nothing to upgrade. Well, I guess I could use a USB hub. But the ERL’s USB port is inside the case. With all the screws in. And the tools are neatly put away. And I’d have to pull the USB hub from behind a workstation. And it’s two am. And I cleaned up the cabling in the lab this past weekend. Looks nice for once. So I don’t want to futz around with all that. There must be an almost imperceptibly easier way of doing this than setting up a TFTP server or NFS share in five minutes… Right?

    iXsystems Boise Technology Show 2018 Recap

    OpenZFS User Conference Slides & Videos Batch editing files with ed
    • what’s ‘ed’?

    ed is this sort of terrifying text editor. A typical interaction with ed for me in the past has gone something like this:

    $ ed help ? h ? asdfasdfasdfsadf ? <close terminal in frustration>

    Basically if you do something wrong, ed will just print out a single, unhelpful, ?. So I’d basically dismissed ed as an old arcane Unix tool that had no practical use today. vi is a successor to ed, except with a visual interface instead of this ?

    • surprise: Ed is actually sort of cool and fun

    So if Ed is a terrifying thing that only prints ? at you, why am I writing a blog post about it? WELL!!!! On April 1 this year, Michael W Lucas published a new short book called Ed Mastery. I like his writing, and even though it was sort of an april fool’s joke, it was ALSO a legitimate actual real book, and so I bought it and read it to see if his claims that Ed is actually interesting were true. And it was so cool!!!! I found out:

    • how to get Ed to give you better error messages than just ?
    • that the name of the grep command comes from ed syntax (g/re/p)
    • the basics of how to navigate and edit files using ed

    All of that was a cool Unix history lesson, but did not make me want to actually use Ed in real life. But!!!

    The other neat thing about Ed (that did make me want to use it!) is that any Ed session corresponds to a script that you can replay! So if I know Ed, then I can use Ed basically as a way to easily apply vim-macro-like programs to my files.

    Beastie Bits

    Tarsnap

    Feedback/Questions
    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Thu, 24 May 2018 12:00:00 -0700
    Episode 246: Properly Coordinated Disclosure | BSD Now 246
    How Intel docs were misinterpreted by almost any OS, a look at the mininet SDN emulator, do’s and don’ts for FreeBSD, OpenBSD community going gold, ed mastery is a must read, and the distributed object store minio on FreeBSD. Headlines Intel documentation flaw sees instruction misimplemented in almost every OS

    A statement in the System Programming Guide of the Intel 64 and IA-32 Architectures Software Developer's Manual (SDM) was mishandled in the development of some or all operating-system kernels, resulting in unexpected behavior for #DB exceptions that are deferred by MOV SS or POP SS, as demonstrated by (for example) privilege escalation in Windows, macOS, some Xen configurations, or FreeBSD, or a Linux kernel crash. OS kernels may not expect this order of events and may therefore experience unexpected behavior when it occurs. + A detailed white paper describes this behavior here + FreeBSD Commit Thank you to the MSRC Incident Response Team, and in particular Greg Lenti and Nate Warfield, for coordinating the response to this issue across multiple vendors. Thanks to Computer Recycling at The Working Center of Kitchener for making hardware available to allow us to test the patch on additional CPU families. + FreeBSD Security Advisory + DragonFlyBSD Post + NetBSD does not support debug register and so is not affected. + OpenBSD also appears to not be affected, “We are not aware of further vendor information regarding this vulnerability.” + IllumOS Not Impacted

    Guest Post – A Look at SDN Emulator Mininet
    • A guest post on the FreeBSD Foundation’s blog by developer Ayaka Koshibe At this year’s AsiaBSDCon, I presented a talk about a SDN network emulator called Mininet, and my ongoing work to make it more portable. That presentation was focused on the OpenBSD version of the port, and I breezed past the detail that I also had a version or Mininet working on FreeBSD. Because I was given the opportunity, I’d like to share a bit about the FreeBSD version of Mininet. It will not only be about what Mininet is and why it might be interesting, but also a recounting of my experience as a user making a first-time attempt at porting an application to FreeBSD. Mininet started off as a tool used by academic researchers to emulate OpenFlow networks when they didn’t have convenient access to actual networks. Because of its history, Mininet became associated strongly with networks that use OpenFlow for their control channels. But, it has also become fairly popular among developers working in, and among several universities for research and teaching about, SDN (Software Defined Networking) I began using Mininet as an intern at my university’s network research lab. I was using FreeBSD by that time, and wasn’t too happy to learn that Mininet wouldn’t work on anything but Linux. I gradually got tired of having to run a Linux VM just to use Mininet, and one day it clicked in my mind that I can actually try porting it to FreeBSD. Mininet creates a topology using the resource virtualization features that Linux has. Specifically, nodes are bash processes running in network namespaces, and the nodes are interconnected using veth virtual Ethernet links. Switches and controllers are just nodes whose shells have run the right commands to configure a software switch or start a controller application. Mininet can therefore be viewed as a series of Python libraries that run the system commands necessary to create network namespaces and veth interfaces, assemble a specified topology, and coordinate how user commands aimed at nodes (since they are just shells) are run. Coming back to the port, I chose to use vnet jails to replace the network namespaces, and epair(4) links to replace the veth links. For the SDN functionality, I needed at least one switch and controller that can be run on FreeBSD. I chose OpenvSwitch(OVS) for the switch, since it was available in ports and is well-known by the SDN world, and Ryu for the controller since it’s being actively developed and used and supports more recent versions of OpenFlow. I have discussed the possibility of upstreaming my work. Although they were excited about it, I was asked about a script for creating VMs with Mininet preinstalled, and continuous integration support for my fork of the repository. I started taking a look at the release scripts for creating a VM, and after seeing that it would be much easier to use the scripts if I can get Mininet and Ryu added to the ports tree, I also tried a hand at submitting some ports. For CI support, Mininet uses Travis, which unfortunately doesn’t support FreeBSD. For this, I plan to look at a minimalistic CI tool called contbuild, which looks simple enough to get running and is written portably. This is very much a work-in-progress, and one going at a glacial pace. Even though the company that I work for does use Mininet, but doesn’t use FreeBSD, so this is something that I’ve been working on in my free time. Earlier on, it was the learning curve that made progress slow. When I started, I hadn’t done anything more than run FreeBSD on a laptop, and uneventfully build a few applications from the ports tree. Right off the bat, using vnet jails meant learning how to build and run a custom kernel. This was the easy part, as the handbook was clear about how to do this. When I moved from using FreeBSD 10.3 to 11, I found that I can panic my machine by quickly creating and destroying OVS switches and jails. I submitted a bug report, but decided to go one step further and actually try to debug the panic for myself. With the help of a few people well-versed in systems programming and the developer’s handbook, I was able to come up with a fix, and get it accepted. This pretty much brings my porting experiment to the present day, where I’m slowly working out the pieces that I mentioned earlier. In the beginning, I thought that this Mininet port would be a weekend project where I come out knowing thing or two about using vnet jails and with one less VM to run. Instead, it became a crash course in building and debugging kernels and submitting bug reports, patches, and ports. It’d like to mention that I wouldn’t have gotten far at all if it weren’t for the helpful folks, the documentation, and how debuggable FreeBSD is. I enjoy good challenges and learning experiences, and this has definitely been both.
    • Thank you to Ayaka for working to port Mininet to the BSDs, and for sharing her experiences with us.
    • If you want to see the OpenBSD version of the talk, the video from AsiaBSDCon is here, and it will be presented again at BSDCan.
    **iXsystems** [iXsystems LFNW Recap](https://www.ixsystems.com/blog/lfnw-2018-recap/) 10 Beginner Do's and Don't for FreeBSD
    • 1) Don't mix ports and binary packages
    • 2) Don't edit 'default' files
    • 3) Don't mess with /etc/crontab
    • 4) Don't mess with /etc/passwd and /etc/groups either!
    • 5) Reconsider the removal of any options from your customized kernel configuration
    • 6) Don't change the root shell to something else
    • 7) Don't use the root user all the time
    • 8) /var/backups is a thing
    • 9) Check system integrity using /etc/mtree
    • 10) What works for me doesn't have to work for you!
    News Roundup OpenBSD Community Goes Gold for 2018!
    • Ken Westerback (krw@ when wearing his developer hat) writes:

    ``` Monthly paypal donations from the OpenBSD community have made the community the OpenBSD Foundation's first Gold level contributor for 2018!

    2018 is the third consecutive year that the community has reached Gold status or better.

    These monthly paypal commitments by the community are our most reliable source of funds and thus the most useful for financial planning purposes. We are extremely thankful for the continuing support and hope the community matches their 2017 achievement of Platinum status. Or even their 2016 achievement of Iridium status.

    Sign up now for a monthly donation!

    Note that Bitcoin contributions have been re-enabled now that our Bitcoin intermediary has re-certified our Canadian paperwork.

    https://www.openbsdfoundation.org/donations.html ```

    ed(1) mastery is a must read for real unix people

    In some circles on the Internet, your choice of text editor is a serious matter.

    We've all seen the threads on mailing lits, USENET news groups and web forums about the relative merits of Emacs vs vi, including endless iterations of flame wars, and sometimes even involving lesser known or non-portable editing environments.

    And then of course, from the Linux newbies we have seen an endless stream of tweeted graphical 'memes' about the editor vim (aka 'vi Improved') versus the various apparently friendlier-to-some options such as GNU nano. Apparently even the 'improved' version of the classical and ubiquitous vi(1) editor is a challenge even to exit for a significant subset of the younger generation.

    Yes, your choice of text editor or editing environment is a serious matter. Mainly because text processing is so fundamental to our interactions with computers.

    But for those of us who keep our systems on a real Unix (such as OpenBSD or FreeBSD), there is no real contest. The OpenBSD base system contains several text editors including vi(1) and the almost-emacs mg(1), but ed(1) remains the standard editor.

    Now Michael Lucas has written a book to guide the as yet uninitiated to the fundamentals of the original Unix text editor. It is worth keeping in mind that much of Unix and its original standard text editor written back when the standard output and default user interface was more likely than not a printing terminal.

    To some of us, reading and following the narrative of Ed Mastery is a trip down memory lane. To others, following along the text will illustrate the horror of the world of pre-graphic computer interfaces. For others again, the fact that ed(1) doesn't use your terminal settings much at all offers hope of fixing things when something or somebody screwed up your system so you don't have a working terminal for that visual editor.

    DigitalOcean Digital Ocean Promo Link for BSD Now Listeners

    Distributed Object Storage with Minio on FreeBSD

    Free and open source distributed object storage server compatible with Amazon S3 v2/v4 API. Offers data protection against hardware failures using erasure code and bitrot detection. Supports highly available distributed setup. Provides confidentiality, integrity and authenticity assurances for encrypted data with negligible performance overhead. Both server side and client side encryption are supported. Below is the image of example Minio setup.

    The Minio identifies itself as the ZFS of Cloud Object Storage. This guide will show You how to setup highly available distributed Minio storage on the FreeBSD operating system with ZFS as backend for Minio data. For convenience we will use FreeBSD Jails operating system level virtualization.

    • Setup

    The setup will assume that You have 3 datacenters and assumption that you have two datacenters in whose the most of the data must reside and that the third datacenter is used as a ‘quorum/witness’ role. Distributed Minio supports up to 16 nodes/drives total, so we may juggle with that number to balance data between desired datacenters. As we have 16 drives to allocate resources on 3 sites we will use 7 + 7 + 2 approach here. The datacenters where most of the data must reside have 7/16 ratio while the ‘quorum/witness’ datacenter have only 2/16 ratio. Thanks to built in Minio redundancy we may loose (turn off for example) any one of those machines and our object storage will still be available and ready to use for any purpose.

    • Jails

    First we will create 3 jails for our proof of concept Minio setup, storage1 will have the ‘quorum/witness’ role while storage2 and storage3 will have the ‘data’ role. To distinguish commands I type on the host system and storageX Jail I use two different prompts, this way it should be obvious what command to execute and where.

    • WeI know the FreeNAS people have been working on integrating this
    Best practises for pledge(2) security

    Let's set the record straight for securing kcgi CGI and FastCGI applications with pledge(2). This is focussed on secure OpenBSD deployments.

    • Theory

    Internally, kcgi makes considerable use of available security tools. But it's also designed to be invoked in a secure environment. We'll start with pledge(2), which has been around on OpenBSD since version 5.9. If you're reading this tutorial, you're probably on OpenBSD, and you probably have knowledge of pledge(2).

    How to begin? Read kcgi(3). It includes canonical information on which pledge(2) promises you'll need for each function in the library. This is just a tutorial—the manpage is canonical and overrides what you may read here.

    Next, assess the promises that your application needs. From kcgi(3), it's easy to see which promises we'll need to start. You'll need to augment this list with whichever tools you're also using. The general push is to start with the broadest set of required promises, then restrict as quickly as possible. Sometimes this can be done in a single pledge(2), but other times it takes a few.

    Beastie Bits Feedback/Questions
    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Wed, 16 May 2018 22:00:00 -0700
    Episode 245: ZFS User Conf 2018 | BSD Now 245
    Allan’s recap of the ZFS User conference, first impressions of OmniOS by a BSD user, Nextcloud 13 setup on FreeBSD, OpenBSD on a fanless desktop computer, an intro to HardenedBSD, and DragonFlyBSD getting some SMP improvements. Headlines ZFS User Conference Recap
    • Attendees met for breakfast on the fourth floor, in a lunchroom type area just outside of the theatre. One entire wall was made of lego base plates, and there were buckets of different coloured lego embedded in the wall.
    • The talks started with Matt Ahrens discussing how the 2nd most requested feature of ZFS, Device Removal, has now landed, then pivoting into the MOST requested feature, RAID-Z expansion, and his work on that so far, which included the first functional prototype, on FreeBSD.
    • Then our friend Calvin Hendryx-Parker presented how he solves all of his backup headaches with ZFS. I provided him some helpful hints to optimize his setup and improve the throughput of his backups
    • Then Steven Umbehocker of OSNEXUS talked about their products, and how they manage large numbers of ZFS nodes
    • After a very nice lunch, Orlando Pichardo of Micron talked about the future of flash, and their new 7.5TB SATA SSDs. Discussion of these devices after the talk may lead to enhancements to ZFS to better support these new larger flash devices that use larger logical sector sizes.
    • Alek Pinchuk of Datto talked about Pool Layout Considerations
    • then Tony Hutter of LLNL talked about the release process for ZFS on Linux
    • Then Tom Caputi of Datto presented: Helping Developers Help You, guidance for users submitting bug reports, with some good and bad examples
    • Then we had a nice cocktail party and dinner, and stayed late into the night talked about ZFS
    • The next day, Jervin Real of Percona, presented: ZFS and MySQL on Linux, the Sweet Spots. Mostly outlining some benchmark they had done, some of the results were curious and some additional digging may turn up enhancements that can be made to ZFS, or just better tuning advice for high traffic MySQL servers.
    • Then I presented my ZSTD compression work, which had been referenced in 2 of the previous talks, as people are anxious to get their hands on this code.
    • Lastly, Eric Sproul of Circonus, gave his talk: Thank You, ZFS. It thanked ZFS and its Community for making their companies product possible, and then provided an update to his presentation from last year, where they were having problems with extremely high levels of ZFS fragmentation. This also sparked a longer conversation after the talk was over.
    • Then we had a BBQ lunch, and after some more talking, the conference broke up.
    Initial OmniOS impressions by a BSD user

    I had been using FreeBSD as my main web server OS since 2012 and I liked it so much that I even contributed money and code to it. However, since the FreeBSD guys (and gals) decided to install anti-tech feminism, I have been considering to move away from it for quite some time now.

    As my growing needs require stronger hardware, it was finally time to rent a new server. I do not intend to run FreeBSD on it. Although the most obvious choice would be OpenBSD (I run it on another server and it works just fine), I plan to have a couple of databases running on the new machine, and database throughput has never been one of OpenBSD's strong points. This is my chance to give illumos another try. As neither WiFi nor desktop environments are relevant on a no-X11 server, the server-focused OmniOS seemed to fit my needs.

    My current (to be phased out) setup on FreeBSD is:

    • apache24 with SSL support, running five websites on six domains (both HTTP and HTTPS)
    • a (somewhat large) Tiny Tiny RSS installation from git, updated via cronjob
    • sbcl running a daily cronjob of my Web-to-RSS parser
    • an FTP server where I share stuff with friends
    • an IRC bouncer
    • MariaDB and PostgreSQL for some of the hosted services

    I would not consider anything of that too esoteric for a modern operating system. Since I was not really using anything mod_rewrite-related, I was perfectly ready to replace apache24 by nginx, remembering that the prepackaged apache24 on FreeBSD did not support HTTPS out of the box and I had ended up installing it from the ports. That is the only change in my setup which I am actively planning.

    So here's what I noticed.

    • First impressions:

    Hooray, a BSD boot loader! Finally an operating system without grub - I made my experiences with that and I don't want to repeat them too often.

    It is weird that the installer won't accept "mydomain.org" as a hostname but sendmail complains that "mydomain" is not a valid hostname right from the start, OmniOS sent me into Maintenance Mode to fix that. A good start, right? So the first completely new thing I had to find out on my new shiny toy was how to change the hostname. There is no /etc/rc.conf in it and hostname mydomain.org was only valid for one login session. I found out that the hostname has to be changed in three different files under /etc on Solaris - the third one did not even exist for me. Changing the other two files seems to have solved this problem for me.

    • Random findings:

    ~ I was wondering how many resources my (mostly idle) new web server was using - I always thought Solaris was rather fat, but it still felt fast to me.

    Ah, right - we're in Unixland and we need to think outside of the box. This table was really helpful: although a number of things are different between OmniOS and SmartOS, I found out that the *stat tools do what top does. I could probably just install top from one of the package managers, but I failed to find a reason to do so. I had 99% idle CPU and RAM - that's all I wanted to know.

    ~ Trying to set up twtxt informed me that Python 3.6 (from pkgin) expects LANG and LC_ALL to be set. Weird - did FreeBSD do that for me? It's been a while ... at least that was easy to fix.

    ~ SMF - Solaris's version of init - confuses me. It has "levels" similar to Gentoo's OpenRC, but it mostly shuts up during the boot process. Stuff from pkgsrc, e.g. nginx, comes with a description how to set up the particular service, but I should probably read more about it. What if, one day, I install a package which is not made ready for OmniOS? I'll have to find out how to write SMF scripts. But that should not be my highest priority.

    ~ The OmniOS documentation talks a lot about "zones" which, if I understand that correctly, mostly equal FreeBSD's "jails". This could be my chance to try to respect a better separation between my various services - if my lazyness won't take over again. (It probably will.)

    ~ OmniOS's default shell - rather un-unixy - seems to be the bash. Update: I was informed about a mistake here: the default shell is ksh93, there are bogus .bashrc files lying around though.

    ~ Somewhere in between, my sshd had a hiccup or, at least, logging into it took longer than usual. If that happens again, I should investigate.

    • Conclusion:

    By the time of me writing this, I have a basic web server with an awesome performance and a lot of applications ready to be configured only one click away. The more I play with it, the more I have the feeling that I have missed a lot while wasting my time with FreeBSD. For a system that is said to be "dying", OmniOS feels well-thought and, when equipped with a reasonable package management, comes with everything I need to reproduce my FreeBSD setup without losing functionality.

    I'm looking forward to what will happen with it.

    DigitalOcean http://do.co/bsdnow

    [Open Source Hardware Camp 2018 — Sat 30/06 & Sun 01/07, Lincoln, UK

    (includes 'Open-source RISC-V core quickstart' and 'An introductory workshop to NetBSD on embedded platforms')](http://oshug.org/pipermail/oshug/2018-April/000635.html)

    ``` Hi All,

    I'm pleased to announce that we have 10 talks and 7 workshops confirmed for Open Source Hardware Camp 2018, with the possibility of one or two more. Registration is now open!

    For the first time ever we will be hosting OSHCamp in Lincoln and a huge thanks to Sarah Markall for helping to make this happen.

    As in previous years, there will be a social event on the Saturday evening and we have a room booked at the Wig and Mitre. Food will be available.

    There will likely be a few of us meeting up for pre-conference drinks on the Friday evening also.

    Details of the programme can be found below and, as ever, we have an excellent mix of topics being covered.

    Cheers,

    Andrew ```

    • Open Source Hardware Camp 2018

    On the 30th June 2018, 09:00 Saturday morning - 16:00 on the Sunday afternoon at The Blue Room, The Lawn, Union Rd, Lincoln, LN1 3BU.

    • Registration: http://oshug.org/event/oshcamp2018
    • Open Source Hardware Camp 2018 will be hosted in the historic county town of Lincoln — home to, amongst others, noted engine builders Ruston & Hornsby (now Siemens, via GEC and English Electric).
    • Lincoln is well served by rail, reachable from Leeds and London within 2-2.5 hours, and 4-5 hours from Edinburgh and Southampton.
    • There will be a social at the Wig and Mitre on the Saturday evening.
    • For travel and accommodation information information please see the event page on oshug.org.
    News Roundup Nextcloud 13 on FreeBSD

    Today I would like to share a setup of Nextcloud 13 running on a FreeBSD system. To make things more interesting it would be running inside a FreeBSD Jail. I will not describe the Nextcloud setup itself here as its large enough for several blog posts.

    Official Nextcloud 13 documentation recommends following setup:

    • MySQL/MariaDB
    • PHP 7.0 (or newer)
    • Apache 2.4 (with mod_php)

    I prefer PostgreSQL database to MySQL/MariaDB and I prefer fast and lean Nginx web server to Apache, so my setup is based on these components:

    • PostgreSQL 10.3
    • PHP 7.2.4
    • Nginx 1.12.2 (with php-fpm)
    • Memcached 1.5.7

    The Memcached subsystem is least important, it can be easily changed into something more modern like Redis for example. I prefer not to use any third party tools for FreeBSD Jails management. Not because they are bad or something like that. There are just many choices for good FreeBSD Jails management and I want to provide a GENERIC example for Nextcloud 13 in a Jail, not for a specific management tool.

    • Host

    Lets start with preparing the FreeBSD Host with needed settings. We need to allow using raw sockets in Jails. For the future optional upgrades of the Jail we will also allow using chflags(1) in Jails.

    OpenBSD on my fanless desktop computer

    You asked me about my setup. Here you go.

    I’ve been using OpenBSD on servers for years as a web developer, but never had a chance to dive in to system administration before. If you appreciate the simplicity of OpenBSD and you have to give it a try on your desktop.

    Bear in mind, this is a relatively cheap ergonomic setup, because all I need is xterm(1) with Vim and Firefox, I don’t care about CPU/GPU performance or mobility too much, but I want a large screen and a good keyboard.

    Item Price, USD Zotac CI527 NANO-BE $371 16GB RAM Crucial DDR4-2133 $127 250GB SSD Samsung 850 EVO $104 Asus VZ249HE 23.8" IPS Full HD $129 ErgoDox EZ V3, Cherry MX Brown, blank DCS $325 Kensington Orbit Trackball $33 Total $1,107

    • OpenBSD

    I tried few times to install OpenBSD on my MacBooks—I heard some models are compatible with it,—but in my case it was a bit of a fiasco (thanks to Nvidia and Broadcom). That’s why I bought a new computer, just to be able to run this wonderful operating system.

    Now I run -stable on my desktop and servers. Servers are supposed to be reliable, that’s obvious, why not run -current on a desktop? Because -stable is shipped every six months and I that’s is often enough for me. I prefer slow fashion.

    iXsystems iX Ad Spot NAB 2018 – Michael Dexter’s Recap

    Introduction to HardenedBSD World

    HardenedBSD is a security enhanced fork of FreeBSD which happened in 2014. HardenedBSD is implementing many exploit mitigation and security technologies on top of FreeBSD which all started with implementation of Address Space Layout Randomization (ASLR). The fork has been created for ease of development.

    To cite the https://hardenedbsd.org/content/about page – “HardenedBSD aims to implement innovative exploit mitigation and security solutions for the FreeBSD community. (…) HardenedBSD takes a holistic approach to security by hardening the system and implementing exploit mitigation technologies.”

    Most FreeBSD enthusiasts know mfsBSD project by Martin Matuska – http://mfsbsd.vx.sk/ – FreeBSD system loaded completely into memory. The mfsBSD synonym for the HardenedBSD world is SoloBSD – http://www.solobsd.org/ – which is based on HardenedBSD sources.

    One may ask how HardenedBSD project compared to more well know for its security OpenBSD system and it is very important question. The OpenBSD developers try to write ‘good’ code without dirty hacks for performance or other reasons. Clean and secure code is most important in OpenBSD world. The OpenBSD project even made security audit of all OpenBSD code available, line by line. This was easier to achieve in FreeBSD or HardenedBSD because OpenBSD code base its about ten times smaller. This has also other implications, possibilities. While FreeBSD (and HardenedBSD) offer many new features like mature SMP subsystem even with some NUMA support, ZFS filesystem, GEOM storage framework, Bhyve virtualization, Virtualbox option and many other new modern features the OpenBSD remains classic UNIX system with UFS filesystem and with very ‘theoretical’ SMP support. The vmm project tried to implement new hypervisor in OpenBSD world, but because of lack of support for graphics its for OpenBSD, Illumos and Linux currently, You will not virtualize Windows or Mac OS X there. This is also only virtualization option for OpenBSD as there are no Jails on OpenBSD. Current Bhyve implementation allows one even to boot latest Windows 2019 Technology Preview.

    A HardenedBSD project is FreeBSD system code base with LOTS of security mechanisms and mitigations that are not available on FreeBSD system. For example entire lib32 tree has been disabled by default on HardenedBSD to make it more secure. Also LibreSSL is the default SSL library on HardenedBSD, same as OpenBSD while FreeBSD uses OpenSSL for compatibility reasons.

    Comparison between LibreSSL and OpenSSL vulnerabilities.

    • https://en.wikipedia.org/wiki/LibreSSL#Security
    • https://wiki.freebsd.org/LibreSSL#LibreSSL.28andOpenSSL.29SecurityVulnerabilities

    One may see HardenedBSD as FreeBSD being successfully pulled up to the OpenBSD level (at least that is the goal), but as FreeBSD has tons more code and features it will be harder and longer process to achieve the goal.

    As I do not have that much competence on the security field I will just repost the comparison from the HardenedBSD project versus other BSD systems. The comparison is also available here – https://hardenedbsd.org/content/easy-feature-comparison – on the HardenedBSD website.

    Running my own git server

    Note: This article is predominantly based on work by Hiltjo Posthuma who you should read because I would have spent far too much time failing to set things up if it wasn’t for their post. Not only have they written lots of very interesting posts, they write some really brilliant programs

    Since I started university 3 years ago, I started using lots of services from lots of different companies. The “cloud” trend led me to believe that I wanted other people to look after my data for me. I was wrong. Since finding myself loving the ethos of OpenBSD, I found myself wanting to apply this ethos to the services I use as well. Not only is it important to me because of the security benefits, but also because I like the minimalist style OpenBSD portrays. This is the first in a mini-series documenting my move from bloated, hosted, sometimes proprietary services to minimal, well-written, free, self-hosted services.

    • Tools & applications

    These are the programs I am going to be using to get my git server up and running:

    httpd(8) acme-client(1) git(1) cgit(1) slowcgi(8)

    • Setting up httpd

    Ensure you have the necessary flags enabled in your /etc/rc.conf.local:

    • Configuring cgit

    When using the OpenBSD httpd(8), it will serve it’s content in a chrooted environment,which defaults to the home directory of the user it runs as, which is www in this case. This means that the chroot is limited to the directory /var/www and it’s contents.

    In order to configure cgit, there must be a cgitrc file available to cgit. This is found at the location stored in $CGIT_CONFIG, which defaults to /conf/cgitrc. Because of the chroot, this file is actually stored at /var/www/conf/cgitrc.

    Beastie Bits

    Tarsnap ad

    Feedback/Questions
    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Thu, 10 May 2018 05:00:00 -0700
    Episode 244: C is a Lie | BSD Now 244
    Arcan and OpenBSD, running OpenBSD 6.3 on RPI 3, why C is not a low-level language, HardenedBSD switching back to OpenSSL, how the Internet was almost broken, EuroBSDcon CfP is out, and the BSDCan 2018 schedule is available. Headlines Towards Secure System Graphics: Arcan and OpenBSD

    Let me preface this by saying that this is a (very) long and medium-rare technical article about the security considerations and minutiae of porting (most of) the Arcan ecosystem to work under OpenBSD. The main point of this article is not so much flirting with the OpenBSD crowd or adding further noise to software engineering topics, but to go through the special considerations that had to be taken, as notes to anyone else that decides to go down this overgrown and lonesome trail, or are curious about some less than obvious differences between how these things “work” on Linux vs. other parts of the world.

    A disclaimer is also that most of this have been discovered by experimentation and combining bits and pieces scattered in everything from Xorg code to man pages, there may be smarter ways to solve some of the problems mentioned – this is just the best I could find within the time allotted. I’d be happy to be corrected, in patch/pull request form that is 😉

    Each section will start with a short rant-like explanation of how it works in Linux, and what the translation to OpenBSD involved or, in the cases that are still partly or fully missing, will require. The topics that will be covered this time are:

    • Graphics Device Access
    • Hotplug
    • Input
    • Backlight
    • Xorg
    • Pledging
    • Missing
    Installing OpenBSD 6.3 (snapshots) on Raspberry pi 3
    • The Easy way

    Installing the OpenBSD on raspberry pi 3 is very easy and well documented which almost convinced me of not writing about it, but still I felt like it may help somebody new to the project (But again I really recommend reading the document if you are interested and have the time).

    Note: I'm always running snapshots and recommend anybody to do it as well. But the snapshots links will change to the next version every 6 month, so I changed the links to the 6.3 version to keep the blog post valid over times. If you're familiar to the OpenBSD flavors, feel free to use the snapshots links instead.

    • Requirements

    Due to the lack of driver, the OpenBSD can not boot directly from the SD Card yet, So we'll need an USB Stick for the installtion target aside the SD Card for the U-Boot and installer. Also, a Serial Console connection is required. I Used a PL2303 USB to Serial (TTL) adapter connected to my Laptop via USB port and connected to the Raspberry via TX, RX and GND pins.

    iXsystems https://www.ixsystems.com/blog/truenas-m-series-veeam-pr-2018/

    Why Didn’t Larrabee Fail?

    Every month or so, someone will ask me what happened to Larrabee and why it failed so badly. And I then try to explain to them that not only didn't it fail, it was a pretty huge success. And they are understandably very puzzled by this, because in the public consciousness Larrabee was like the Itanic and the SPU rolled into one, wasn't it? Well, not quite. So rather than explain it in person a whole bunch more times, I thought I should write it down.

    This is not a history, and I'm going to skip a TON of details for brevity. One day I'll write the whole story down, because it's a pretty decent escapade with lots of fun characters. But not today. Today you just get the very start and the very end.

    When I say "Larrabee" I mean all of Knights, all of MIC, all of Xeon Phi, all of the "Isle" cards - they're all exactly the same chip and the same people and the same software effort. Marketing seemed to dream up a new codeword every week, but there was only ever three chips:

    • Knights Ferry / Aubrey Isle / LRB1 - mostly a prototype, had some performance gotchas, but did work, and shipped to partners.
    • Knights Corner / Xeon Phi / LRB2 - the thing we actually shipped in bulk.
    • Knights Landing - the new version that is shipping any day now (mid 2016).

    That's it. There were some other codenames I've forgotten over the years, but they're all of one of the above chips. Behind all the marketing smoke and mirrors there were only three chips ever made (so far), and only four planned in total (we had a thing called LRB3 planned between KNC and KNL for a while). All of them are "Larrabee", whether they do graphics or not.

    When Larrabee was originally conceived back in about 2005, it was called "SMAC", and its original goals were, from most to least important:

    • Make the most powerful flops-per-watt machine for real-world workloads using a huge array of simple cores, on systems and boards that could be built into bazillo-core supercomputers.
    • Make it from x86 cores. That means memory coherency, store ordering, memory protection, real OSes, no ugly scratchpads, it runs legacy code, and so on. No funky DSPs or windowed register files or wacky programming models allowed. Do not build another Itanium or SPU!
    • Make it soon. That means keeping it simple.
    • Support the emerging GPGPU market with that same chip. Intel were absolutely not going to build a 150W PCIe card version of their embedded graphics chip (known as "Gen"), so we had to cover those programming models. As a bonus, run normal graphics well.
    • Add as little graphics-specific hardware as you can get away with.

    That ordering is important - in terms of engineering and focus, Larrabee was never primarily a graphics card. If Intel had wanted a kick-ass graphics card, they already had a very good graphics team begging to be allowed to build a nice big fat hot discrete GPU - and the Gen architecture is such that they'd build a great one, too. But Intel management didn't want one, and still doesn't. But if we were going to build Larrabee anyway, they wanted us to cover that market as well.

    ... the design of Larrabee was of a CPU with a very wide SIMD unit, designed above all to be a real grown-up CPU - coherent caches, well-ordered memory rules, good memory protection, true multitasking, real threads, runs Linux/FreeBSD, etc. Larrabee, in the form of KNC, went on to become the fastest supercomputer in the world for a couple of years, and it's still making a ton of money for Intel in the HPC market that it was designed for, fighting very nicely against the GPUs and other custom architectures. Its successor, KNL, is just being released right now (mid 2016) and should do very nicely in that space too. Remember - KNC is literally the same chip as LRB2. It has texture samplers and a video out port sitting on the die. They don't test them or turn them on or expose them to software, but they're still there - it's still a graphics-capable part.

    But it's still actually running FreeBSD on that card, and under FreeBSD it's just running an x86 program called DirectXGfx (248 threads of it).

    News Roundup C Is Not a Low-level Language : Your computer is not a fast PDP-11.

    In the wake of the recent Meltdown and Spectre vulnerabilities, it's worth spending some time looking at root causes. Both of these vulnerabilities involved processors speculatively executing instructions past some kind of access check and allowing the attacker to observe the results via a side channel. The features that led to these vulnerabilities, along with several others, were added to let C programmers continue to believe they were programming in a low-level language, when this hasn't been the case for decades.

    Processor vendors are not alone in this. Those of us working on C/C++ compilers have also participated.

    • What Is a Low-Level Language?

    Computer science pioneer Alan Perlis defined low-level languages this way: "A programming language is low level when its programs require attention to the irrelevant."

    While, yes, this definition applies to C, it does not capture what people desire in a low-level language. Various attributes cause people to regard a language as low-level. Think of programming languages as belonging on a continuum, with assembly at one end and the interface to the Starship Enterprise's computer at the other. Low-level languages are "close to the metal," whereas high-level languages are closer to how humans think.

    For a language to be "close to the metal," it must provide an abstract machine that maps easily to the abstractions exposed by the target platform. It's easy to argue that C was a low-level language for the PDP-11. They both described a model in which programs executed sequentially, in which memory was a flat space, and even the pre- and post-increment operators cleanly lined up with the PDP-11 addressing modes.

    Fast PDP-11 Emulators

    The root cause of the Spectre and Meltdown vulnerabilities was that processor architects were trying to build not just fast processors, but fast processors that expose the same abstract machine as a PDP-11. This is essential because it allows C programmers to continue in the belief that their language is close to the underlying hardware.

    C code provides a mostly serial abstract machine (until C11, an entirely serial machine if nonstandard vendor extensions were excluded). Creating a new thread is a library operation known to be expensive, so processors wishing to keep their execution units busy running C code rely on ILP (instruction-level parallelism). They inspect adjacent operations and issue independent ones in parallel. This adds a significant amount of complexity (and power consumption) to allow programmers to write mostly sequential code. In contrast, GPUs achieve very high performance without any of this logic, at the expense of requiring explicitly parallel programs.

    The quest for high ILP was the direct cause of Spectre and Meltdown. A modern Intel processor has up to 180 instructions in flight at a time (in stark contrast to a sequential C abstract machine, which expects each operation to complete before the next one begins). A typical heuristic for C code is that there is a branch, on average, every seven instructions. If you wish to keep such a pipeline full from a single thread, then you must guess the targets of the next 25 branches. This, again, adds complexity; it also means that an incorrect guess results in work being done and then discarded, which is not ideal for power consumption. This discarded work has visible side effects, which the Spectre and Meltdown attacks could exploit.

    On a modern high-end core, the register rename engine is one of the largest consumers of die area and power. To make matters worse, it cannot be turned off or power gated while any instructions are running, which makes it inconvenient in a dark silicon era when transistors are cheap but powered transistors are an expensive resource. This unit is conspicuously absent on GPUs, where parallelism again comes from multiple threads rather than trying to extract instruction-level parallelism from intrinsically scalar code. If instructions do not have dependencies that need to be reordered, then register renaming is not necessary.

    Consider another core part of the C abstract machine's memory model: flat memory. This hasn't been true for more than two decades. A modern processor often has three levels of cache in between registers and main memory, which attempt to hide latency.

    The cache is, as its name implies, hidden from the programmer and so is not visible to C. Efficient use of the cache is one of the most important ways of making code run quickly on a modern processor, yet this is completely hidden by the abstract machine, and programmers must rely on knowing implementation details of the cache (for example, two values that are 64-byte-aligned may end up in the same cache line) to write efficient code.

    HardenedBSD Switching Back to OpenSSL

    Over a year ago, HardenedBSD switched to LibreSSL as the default cryptographic library in base for 12-CURRENT. 11-STABLE followed suit later on. Bernard Spil has done an excellent job at keeping our users up-to-date with the latest security patches from LibreSSL.

    After recently updating 12-CURRENT to LibreSSL 2.7.2 from 2.6.4, it has become increasingly clear to us that performing major upgrades requires a team larger than a single person. Upgrading to 2.7.2 caused a lot of fallout in our ports tree. As of 28 Apr 2018, several ports we consider high priority are still broken. As it stands right now, it would take Bernard a significant amount of his spare personal time to fix these issues.

    Until we have a multi-person team dedicated to maintaining LibreSSL in base along with the patches required in ports, HardenedBSD will use OpenSSL going forward as the default cryptographic library in base. LibreSSL will co-exist with OpenSSL in the source tree, as it does now. However, MK_LIBRESSL will default to "no" instead of the current "yes". Bernard will continue maintaining LibreSSL in base along with addressing the various problematic ports entries.

    To provide our users with ample time to plan and perform updates, we will wait a period of two months prior to making the switch. The switch will occur on 01 Jul 2018 and will be performed simultaneously in 12-CURRENT and 11-STABLE. HardenedBSD will archive a copy of the LibreSSL-centric package repositories and binary updates for base for a period of six months after the switch (expiring the package repos on 01 Jan 2019). This essentially gives our users eight full months for an upgrade path.

    As part of the switch back to OpenSSL, the default NTP daemon in base will switch back from OpenNTPd to ISC NTP. Users who have localopenntpdenable="YES" set in rc.conf will need to switch back to ntpd_enable="YES".

    Users who build base from source will want to fully clean their object directories. Any and all packages that link with libcrypto or libssl will need to be rebuilt or reinstalled.

    With the community's help, we look forward to the day when we can make the switch back to LibreSSL. We at HardenedBSD believe that providing our users options to rid themselves of software monocultures can better increase security and manage risk.

    DigitalOcean http://do.co/bsdnow -- $100 credit for 60 days

    How Dan Kaminsky Almost Broke the Internet

    In the summer of 2008, security researcher Dan Kaminsky disclosed how he had found a huge flaw in the Internet that could let attackers redirect web traffic to alternate servers and disrupt normal operations. In this Hacker History video, Kaminsky describes the flaw and notes the issue remains unfixed.

    “We were really concerned about web pages and emails 'cause that’s what you get to compromise when you compromise DNS,” Kaminsky says. “You think you’re sending an email to IBM but it really goes to the bad guy.”

    As the phone book of the Internet, DNS translates easy-to-remember domain names into IP addresses so that users don’t have to remember strings of numbers to reach web applications and services. Authoritative nameservers publish the IP addresses of domain names. Recursive nameservers talk to authoritative servers to find addresses for those domain names and saves the information into its cache to speed up the response time the next time it is asked about that site. While anyone can set up a nameserver and configure an authoritative zone for any site, if recursive nameservers don’t point to it to ask questions, no one will get those wrong answers.

    We made the Internet less flammable.

    Kaminsky found a fundamental design flaw in DNS that made it possible to inject incorrect information into the nameserver's cache, or DNS cache poisoning. In this case, if an attacker crafted DNS queries looking for sibling names to existing domains, such as 1.example.com, 2.example.com, and 3.example.com, while claiming to be the official "www" server for example.com, the nameserver will save that server IP address for “www” in its cache.

    “The server will go, ‘You are the official. Go right ahead. Tell me what it’s supposed to be,’” Kaminsky says in the video.

    Since the issue affected nearly every DNS server on the planet, it required a coordinated response to address it. Kaminsky informed Paul Vixie, creator of several DNS protocol extensions and application, and Vixie called an emergency summit of major IT vendors at Microsoft’s headquarters to figure out what to do.

    The “fix” involved combining the 16-bit transaction identifier that DNS lookups used with UDP source ports to create 32-bit transaction identifiers. Instead of fixing the flaw so that it can’t be exploited, the resolution focused on making it take more than ten seconds, eliminating the instantaneous attack.

    “[It’s] not like we repaired DNS,” Kaminsky says. “We made the Internet less flammable.”

    DNSSEC (Domain Name System Security Extensions), is intended to secure DNS by adding a cryptographic layer to DNS information. The root zone of the internet was signed for DNSSEC in July 2010 and the .com Top Level Domain (TLD) was finally signed for DNSSEC in April 2011. Unfortunately, adoption has been slow, even ten years after Kaminsky first raised the alarm about DNS, as less than 15 percent of users pass their queries to DNSSEC validating resolvers.

    The Internet was never designed to be secure. The Internet was designed to move pictures of cats.

    No one expected the Internet to be used for commerce and critical communications. If people lose faith in DNS, then all the things that depend on it are at risk.

    “What are we going to do? Here is the answer. Some of us gotta go out fix it,” Kaminsky says.

    OpenIndiana Hipster 2018.04 is here
    • We have released a new OpenIndiana Hipster snapshot 2018.04. The noticeable changes:

      • Userland software is rebuilt with GCC 6.
      • KPTI was enabled to mitigate recent security issues in Intel CPUs.
      • Support of Gnome 2 desktop was removed.
      • Linked images now support zoneproxy service.
      • Mate desktop applications are delivered as 64-bit-only.
      • Upower support was integrated.
      • IIIM was removed.
    • More information can be found in 2018.04 Release notes and new medias can be downloaded from http://dlc.openindiana.org.

    Beastie Bits

    Tarsnap ad

    Feedback/Questions
    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

    iX Ad spot: iXsystems TrueNAS M-Series Blows Away Veeam Backup Certification Tests

    Thu, 03 May 2018 00:00:00 -0700
    Episode 243: Understanding The Scheduler | BSD Now 243
    OpenBSD 6.3 and DragonflyBSD 5.2 are released, bug fix for disappearing files in OpenZFS on Linux (and only Linux), understanding the FreeBSD CPU scheduler, NetBSD on RPI3, thoughts on being a committer for 20 years, and 5 reasons to use FreeBSD in 2018. Headlines OpenBSD 6.3 released
    • Punctual as ever, OpenBSD 6.3 has been releases with the following features/changes: Improved HW support, including: SMP support on OpenBSD/arm64 platforms vmm/vmd improvements: IEEE 802.11 wireless stack improvements Generic network stack improvements Installer improvements Routing daemons and other userland network improvements Security improvements dhclient(8) improvements Assorted improvements OpenSMTPD 6.0.4 OpenSSH 7.7 LibreSSL 2.7.2
    DragonFlyBSD 5.2 released

    Big-ticket items Meltdown and Spectre mitigation support Meltdown isolation and spectre mitigation support added. Meltdown mitigation is automatically enabled for all Intel cpus. Spectre mitigation must be enabled manually via sysctl if desired, using sysctls machdep.spectremitigation and machdep.meltdownmitigation. HAMMER2 H2 has received a very large number of bug fixes and performance improvements. We can now recommend H2 as the default root filesystem in non-clustered mode. Clustered support is not yet available. ipfw Updates Implement state based "redirect", i.e. without using libalias. ipfw now supports all possible ICMP types. Fix ICMPMAXTYPE assumptions (now 40 as of this release). Improved graphics support The drm/i915 kernel driver has been updated to support Intel Coffeelake GPUs Add 24-bit pixel format support to the EFI frame buffer code. Significantly improve fbio support for the "scfb" XOrg driver. This allows EFI frame buffers to be used by X in situations where we do not otherwise support the GPU. Partly implement the FBIOBLANK ioctl for display powersaving. Syscons waits for drm modesetting at appropriate places, avoiding races. + For more details, check out the “All changes since DragonFly 5.0” section.

    ZFS on Linux bug causes files to disappear
    • A bug in ZoL 0.7.7 caused 0.7.8 to be released just 3 days after the release
    • The bug only impacts Linux, the change that caused the problem was not upstreamed yet, so does not impact ZFS on illumos, FreeBSD, OS X, or Windows
    • The bug can cause files being copied into a directory to not be properly linked to the directory, so they will no longer be listed in the contents of the directory
    • ZoL developers are working on a tool to allow you to recover the data, since no data was actually lost, the files were just not properly registered as part of the directory
    • The bug was introduced in a commit made in February, that attempted to improve performance of datasets created with the case insensitivity option. In an effort to improve performance, they introduced a limit to cap to give up (return ENOSPC) if growing the directory ZAP failed twice.
    • The ZAP is the key-value pair data structure that contains metadata for a directory, including a hash table of the files that are in a directory. When a directory has a large number of files, the ZAP is converted to a FatZAP, and additional space may need to be allocated as additional files are added. Commit cc63068 caused ENOSPC error when copy a large amount of files between two directories. The reason is that the patch limits zap leaf expansion to 2 retries, and return ENOSPC when failed.
    • Finding the root cause of this issue was somewhat hampered by the fact that many people were not able to reproduce the issue. It turns out this was caused by an entirely unrelated change to GNU coreutils.
    • On later versions of GNU Coreutils, the files were returned in a sorted order, resulting in them hitting different buckets in the hash table, and not tripping the retry limit
    • Tools like rsync were unaffected, because they always sort the files before copying
    • If you did not see any ENOSPC errors, you were likely not impacted The intent for limiting retries is to prevent pointlessly growing table to max size when adding a block full of entries with same name in different case in mixed mode. However, it turns out we cannot use any limit on the retry. When we copy files from one directory in readdir order, we are copying in hash order, one leaf block at a time. Which means that if the leaf block in source directory has expanded 6 times, and you copy those entries in that block, by the time you need to expand the leaf in destination directory, you need to expand it 6 times in one go. So any limit on the retry will result in error where it shouldn't.
    • Recommendations for Users from Ryan Yao: The regression makes it so that creating a new file could fail with ENOSPC after which files created in that directory could become orphaned. Existing files seem okay, but I have yet to confirm that myself and I cannot speak for what others know. It is incredibly difficult to reproduce on systems running coreutils 8.23 or later. So far, reports have only come from people using coreutils 8.22 or older. The directory size actually gets incremented for each orphaned file, which makes it wrong after orphan files happen. We will likely have some way to recover the orphaned files (like ext4’s lost+found) and fix the directory sizes in the very near future. Snapshots of the damaged datasets are problematic though. Until we have a subcommand to fix it (not including the snapshots, which we would have to list), the damage can be removed from a system that has it either by rolling back to a snapshot before it happened or creating a new dataset with 0.7.6 (or another release other than 0.7.7), moving everything to the new dataset and destroying the old. That will restore things to pristine condition. It should also be possible to check for pools that are affected, but I have yet to finish my analysis to be certain that no false negatives occur when checking, so I will avoid saying how for now.
    • Writes to existing files cannot trigger this bug, only adding new files to a directory in bulk
    News Roundup des@’s thoughts on being a FreeBSD committer for 20 years

    Yesterday was the twentieth anniversary of my FreeBSD commit bit, and tomorrow will be the twentieth anniversary of my first commit. I figured I’d split the difference and write a few words about it today.

    My level of engagement with the FreeBSD project has varied greatly over the twenty years I’ve been a committer. There have been times when I worked on it full-time, and times when I did not touch it for months. The last few years, health issues and life events have consumed my time and sapped my energy, and my contributions have come in bursts. Commit statistics do not tell the whole story, though: even when not working on FreeBSD directly, I have worked on side projects which, like OpenPAM, may one day find their way into FreeBSD.

    My contributions have not been limited to code. I was the project’s first Bugmeister; I’ve served on the Security Team for a long time, and have been both Security Officer and Deputy Security Officer; I managed the last four Core Team elections and am doing so again this year.

    In return, the project has taught me much about programming and software engineering. It taught me code hygiene and the importance of clarity over cleverness; it taught me the ins and outs of revision control; it taught me the importance of good documentation, and how to write it; and it taught me good release engineering practices.

    Last but not least, it has provided me with the opportunity to work with some of the best people in the field. I have the privilege today to count several of them among my friends.

    For better or worse, the FreeBSD project has shaped my career and my life. It set me on the path to information security in general and IAA in particular, and opened many a door for me. I would not be where I am now without it.

    I won’t pretend to be able to tell the future. I don’t know how long I will remain active in the FreeBSD project and community. It could be another twenty years; or it could be ten, or five, or less. All I know is that FreeBSD and I still have things to teach each other, and I don’t intend to call it quits any time soon.

    iXsystems unveils new TrueNAS M-Series Unified Storage Line

    San Jose, Calif., April 10, 2018 — iXsystems, the leader in Enterprise Open Source servers and software-defined storage, announced the TrueNAS M40 and M50 as the newest high-performance models in its hybrid, unified storage product line. The TrueNAS M-Series harnesses NVMe and NVDIMM to bring all-flash array performance to the award-winning TrueNAS hybrid arrays. It also includes the Intel® Xeon® Scalable Family of Processors and supports up to 100GbE and 32Gb Fibre Channel networking. Sitting between the all-flash TrueNAS Z50 and the hybrid TrueNAS X-Series in the product line, the TrueNAS M-Series delivers up to 10 Petabytes of highly-available and flash-powered network attached storage and rounds out a comprehensive product set that has a capacity and performance option for every storage budget.

    • Designed for On-Premises & Enterprise Cloud Environments

    As a unified file, block, and object sharing solution, TrueNAS can meet the needs of file serving, backup, virtualization, media production, and private cloud users thanks to its support for the SMB, NFS, AFP, iSCSI, Fibre Channel, and S3 protocols.

    At the heart of the TrueNAS M-Series is a custom 4U, dual-controller head unit that supports up to 24 3.5” drives and comes in two models, the M40 and M50, for maximum flexibility and scalability. The TrueNAS M40 uses NVDIMMs for write cache, SSDs for read cache, and up to two external 60-bay expansion shelves that unlock up to 2PB in capacity. The TrueNAS M50 uses NVDIMMs for write caching, NVMe drives for read caching, and up to twelve external 60-bay expansion shelves to scale upwards of 10PB. The dual-controller design provides high-availability failover and non-disruptive upgrades for mission-critical enterprise environments.

    By design, the TrueNAS M-Series unleashes cutting-edge persistent memory technology for demanding performance and capacity workloads, enabling businesses to accelerate enterprise applications and deploy enterprise private clouds that are twice the capacity of previous TrueNAS models. It also supports replication to the Amazon S3, BackBlaze B2, Google Cloud, and Microsoft Azure cloud platforms and can deliver an object store using the ubiquitous S3 object storage protocol at a fraction of the cost of the public cloud.

    • Fast

    As a true enterprise storage platform, the TrueNAS M50 supports very demanding performance workloads with up to four active 100GbE ports, 3TB of RAM, 32GB of NVDIMM write cache and up to 15TB of NVMe flash read cache. The TrueNAS M40 and M50 include up to 24/7 and global next-business-day support, putting IT at ease. The modular and tool-less design of the M-Series allows for easy, non-disruptive servicing and upgrading by end-users and support technicians for guaranteed uptime. TrueNAS has US-Based support provided by the engineering team that developed it, offering the rapid response that every enterprise needs.

    • Award-Winning TrueNAS Features

      • Enterprise: Perfectly suited for private clouds and enterprise workloads such as file sharing, backups, M&E, surveillance, and hosting virtual machines.
      • Unified: Utilizes SMB, AFP, NFS for file storage, iSCSI, Fibre Channel and OpenStack Cinder for block storage, and S3-compatible APIs for object storage. Supports every common operating system, hypervisor, and application.
      • Economical: Deploy an enterprise private cloud and reduce storage TCO by 70% over AWS with built-in enterprise-class features such as in-line compression, deduplication, clones, and thin-provisioning.
      • Safe: The OpenZFS file system ensures data integrity with best-in-class replication and snapshotting. Customers can replicate data to the rest of the iXsystems storage lineup and to the public cloud.
      • Reliable: High Availability option with dual hot-swappable controllers for continuous data availability and 99.999% uptime.
      • Familiar: Provision and manage storage with the same simple and powerful WebUI and REST APIs used in all iXsystems storage products, as well as iXsystems’ FreeNAS Software.
      • Certified: TrueNAS has passed the Citrix Ready, VMware Ready, and Veeam Ready certifications, reducing the risk of deploying a virtualized infrastructure.
      • Open: By using industry-standard sharing protocols, the OpenZFS Open Source enterprise file system and FreeNAS, the world’s #1 Open Source storage operating system (and also engineered by iXsystems), TrueNAS is the most open enterprise storage solution on the market.
    • Availability

    The TrueNAS M40 and M50 will be generally available in April 2018 through the iXsystems global channel partner network. The TrueNAS M-Series starts at under $20,000 USD and can be easily expanded using a linear “per terabyte” pricing model. With typical compression, a Petabtye can be stored for under $100,000 USD. TrueNAS comes with an all-inclusive software suite that provides NFS, Windows SMB, iSCSI, snapshots, clones and replication.

    Understanding and tuning the FreeBSD Scheduler

    ``` Occasionally I noticed that the system would not quickly process the tasks i need done, but instead prefer other, longrunning tasks. I figured it must be related to the scheduler, and decided it hates me.

    A closer look shows the behaviour as follows (single CPU):

    Lets run an I/O-active task, e.g, postgres VACUUM that would continuously read from big files (while doing compute as well [1]):

    pool alloc free read write read write cache - - - - - - ada1s4 7.08G 10.9G 1.58K 0 12.9M 0

    Now start an endless loop:

    while true; do :; done

    And the effect is:

    pool alloc free read write read write cache - - - - - - ada1s4 7.08G 10.9G 9 0 76.8K 0

    The VACUUM gets almost stuck! This figures with WCPU in "top":

    PID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND 85583 root 99 0 7044K 1944K RUN 1:06 92.21% bash 53005 pgsql 52 0 620M 91856K RUN 5:47 0.50% postgres

    Hacking on kern.sched.quantum makes it quite a bit better:

    sysctl kern.sched.quantum=1

    kern.sched.quantum: 94488 -> 7874

    pool alloc free read write read write cache - - - - - - ada1s4 7.08G 10.9G 395 0 3.12M 0

    PID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND 85583 root 94 0 7044K 1944K RUN 4:13 70.80% bash 53005 pgsql 52 0 276M 91856K RUN 5:52 11.83% postgres

    Now, as usual, the "root-cause" questions arise: What exactly does this "quantum"? Is this solution a workaround, i.e. actually something else is wrong, and has it tradeoff in other situations? Or otherwise, why is such a default value chosen, which appears to be ill-deceived?

    The docs for the quantum parameter are a bit unsatisfying - they say its the max num of ticks a process gets - and what happens when they're exhausted? If by default the endless loop is actually allowed to continue running for 94k ticks (or 94ms, more likely) uninterrupted, then that explains the perceived behaviour - buts thats certainly not what a scheduler should do when other procs are ready to run.

    11.1-RELEASE-p7, kern.hz=200. Switching tickless mode on or off does not influence the matter. Starting the endless loop with "nice" does not influence the matter.

    [1] A pure-I/O job without compute load, like "dd", does not show this behaviour. Also, when other tasks are running, the unjust behaviour is not so stongly pronounced. ```

    aarch64 support added

    I have committed about adding initial support for aarch64.

    • booting log on RaspberryPI3:

    ``` boot NetBSD/evbarm (aarch64) Drop to EL1...OK Creating VA=PA tables Creating KSEG tables Creating KVA=PA tables Creating devmap tables MMU Enable...OK VSTART = ffffffc000001ff4 FDT<3ab46000> devmap cpufunc bootstrap consinit ok uboot: args 0x3ab46000, 0, 0, 0

    NetBSD/evbarm (fdt) booting ... FDT /memory [0] @ 0x0 size 0x3b000000 MEM: add 0-3b000000 MEM: res 0-1000 MEM: res 3ab46000-3ab4a000 Usable memory: 1000 - 3ab45fff 3ab4a000 - 3affffff initarm: kernel phys start 1000000 end 17bd000 MEM: res 1000000-17bd000 bootargs: root=axe0 1000 - ffffff 17bd000 - 3ab45fff 3ab4a000 - 3affffff ------------------------------------------ kern_vtopdiff = 0xffffffbfff000000 physical_start = 0x0000000000001000 kernel_start_phys = 0x0000000001000000 kernel_end_phys = 0x00000000017bd000 physical_end = 0x000000003ab45000 VM_MIN_KERNEL_ADDRESS = 0xffffffc000000000 kernel_start_l2 = 0xffffffc000000000 kernel_start = 0xffffffc000000000 kernel_end = 0xffffffc0007bd000 kernel_end_l2 = 0xffffffc000800000 (kernel va area) (devmap va area) VM_MAX_KERNEL_ADDRESS = 0xffffffffffe00000 ------------------------------------------ Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018 The NetBSD Foundation, Inc. All rights reserved. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. NetBSD 8.99.14 (RPI64) #11: Fri Mar 30 12:34:19 JST 2018 ryo@moveq:/usr/home/ryo/tmp/netbsd-src-ryo-wip/sys/arch/evbarm/compile/RPI64 total memory = 936 MB avail memory = 877 MB

    Starting local daemons:. Updating motd. Starting sshd. Starting inetd. Starting cron. The following components reported failures: /etc/rc.d/swap2 See /var/run/rc.log for more information. Fri Mar 30 12:35:31 JST 2018 NetBSD/evbarm (rpi3) (console) login: root Last login: Fri Mar 30 12:30:24 2018 on console rpi3# uname -ap NetBSD rpi3 8.99.14 NetBSD 8.99.14 (RPI64) #11: Fri Mar 30 12:34:19 JST 2018 ryo@moveq:/usr/home/ryo/tmp/netbsd-src-ryo-wip/sys/arch/evbarm/compile/RPI64 evbarm aarch64 rpi3#

    ```

    Now, multiuser mode works stably on fdt based boards (RPI3,SUNXI,TEGRA). But there are still some problems, more time is required for release. also SMP is not yet. See sys/arch/aarch64/aarch64/TODO for more detail. Especially the problems around TLS of rtld, and C++ stack unwindings are too difficult for me to solve, I give up and need someone's help (^o^)/ Since C++ doesn't work, ATF also doesn't work. If the ATF works, it will clarify more issues.

    sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. One evbarm/conf/GENERIC64 kernel binary supports all fdt (bcm2837,sunxi,tegra) based boards. While on 32bit, sys/arch/evbarm/conf/GENERIC will support all fdt based boards...but doesn't work yet. (WIP)

    My deepest appreciation goes to Tohru Nishimura (nisimura@) whose writes vector handlers, context switchings, and so on. and his comments and suggestions were innumerably valuable. I would also like to thank Nick Hudson (skrll@) and Jared McNeill (jmcneill@) whose added support FDT and integrated into evbarm. Finally, I would like to thank Matt Thomas (matt@) whose commited aarch64 toolchains and preliminary support for aarch64.

    Beastie Bits Feedback/Questions
    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Wed, 25 Apr 2018 13:00:00 -0700
    Episode 242: Linux Takes The Fastpath | BSD Now 242
    TrueOS Stable 18.03 released, a look at F-stack, the secret to an open source business model, intro to jails and jail networking, FreeBSD Foundation March update, and the ipsec Errata. Headlines TrueOS STABLE 18.03 Release

    The TrueOS team is pleased to announce the availability of a new STABLE release of the TrueOS project (version 18.03). This is a special release due to the security issues impacting the computing world since the beginning of 2018. In particular, mitigating the “Meltdown” and “Spectre” system exploits make it necessary to update the entire package ecosystem for TrueOS. This release does not replace the scheduled June STABLE update, but provides the necessary and expected security updates for the STABLE release branch of TrueOS, even though this is part-way through our normal release cycle.

    • Important changes between version 17.12 and 18.03

      • “Meltdown” security fixes: This release contains all the fixes to FreeBSD which mitigate the security issues for systems that utilize Intel-based processors when running virtual machines such as FreeBSD jails. Please note that virtual machines or jails must also be updated to a version of FreeBSD or TrueOS which contains these security fixes.
      • “Spectre” security mitigations: This release contains all current mitigations from FreeBSD HEAD for the Spectre memory-isolation attacks (Variant 2). All 3rd-party packages for this release are also compiled with LLVM/Clang 6 (the “retpoline” mitigation strategy). This fixes many memory allocation issues and enforces stricter requirements for code completeness and memory usage within applications. Unfortunately, some 3rd-party applications became unavailable as pre-compiled packages due to non-compliance with these updated standards. These applications are currently being fixed either by the upstream authors or the FreeBSD port maintainers. If there are any concerns about the availability of a critical application for a specific workflow, please search through the changelog of packages between TrueOS 17.12 and 18.03 to verify the status of the application.

    Most systems will need microcode updates for additional Spectre mitigations. The microcode updates are not enabled by default. This work is considered experimental because it is in active development by the upstream vendors. If desired, the microcode updates are available with the new devcpu-data package, which is available in the Appcafe. Install this package and enable the new microcode_update service to apply the latest runtime code when booting the system.

    • Important security-based package updates

      • LibreSSL is updated from version 2.6.3 -> 2.6.4
      • Reminder: LibreSSL is used on TrueOS to build any package which does not explicitly require OpenSSL. All applications that utilize the SSL transport layer are now running with the latest security updates.
      • Browser updates: (Keep in mind that many browsers have also implemented their own security mitigations in the aftermath of the Spectre exploit.)
      • Firefox: 57.0.1 -> 58.0.2
      • Chromium: 61.0.3163.100 -> 63.0.3239.132
      • Qt5 Webengine (QupZilla, Falkon, many others): 5.7.1 -> 5.9.4
    • All pre-compiled packages for this release are built with the latest versions of LLVM/Clang, unless the package explicitly requires GCC. These packages also utilize the latest compile-time mitigations for memory-access security concerns.

    F-Stack

    F-Stack is an user space network development kit with high performance based on DPDK, FreeBSD TCP/IP stack and coroutine API. http://www.f-stack.org

    • Introduction With the rapid development of NIC, the poor performance of data packets processing with Linux kernel has become the bottleneck. However, the rapid development of the Internet needs high performance of network processing, kernel bypass has caught more and more attentions. There are various similar technologies appear, such as DPDK, NETMAP and PF_RING. The main idea of kernel bypass is that Linux is only used to deal with control flow, all data streams are processed in user space. Therefore, kernel bypass can avoid performance bottlenecks caused by kernel packet copying, thread scheduling, system calls and interrupts. Furthermore, kernel bypass can achieve higher performance with multi optimizing methods. Within various techniques, DPDK has been widely used because of its more thorough isolation from kernel scheduling and active community support.

    • F-Stack is an open source network framework with high performance based on DPDK. With following characteristics

      • Ultra high network performance which can achieve network card under full load, 10 million concurrent connections, 5 million RPS, 1 million CPS.
      • Transplant FreeBSD 11.01 user space stack, provides a complete stack function, cut a great amount of irrelevant features. Therefore greatly enhance the performance.
      • Support Nginx, Redis and other mature applications, service can easily use F-Stack
      • With Multi-process architecture, easy to extend
      • Provide micro thread interface. Various applications with stateful app can easily use F-Stack to get high performance without processing complex asynchronous logic.
      • Provide Epoll/Kqueue interface that allow many kinds of applications easily use F-Stack
    • History

    In order to deal with the increasingly severe DDoS attacks, authorized DNS server of Tencent Cloud DNSPod switched from Gigabit Ethernet to 10-Gigabit at the end of 2012. We faced several options, one is to continue to use the original model another is to use kernel bypass technology. After several rounds of investigation, we finally chose to develop our next generation of DNS server based on DPDK. The reason is DPDK provides ultra-high performance and can be seamlessly extended to 40G, or even 100G NIC in the future.

    After several months of development and testing, DKDNS, high-performance DNS server based on DPDK officially released in October 2013. It's capable of achieving up to 11 million QPS with a single 10GE port and 18.2 million QPS with two 10GE ports. And then we developed a user-space TCP/IP stack called F-Stack that can process 0.6 million RPS with a single 10GE port.

    With the fast growth of Tencent Cloud, more and more services need higher network access performance. Meanwhile, F-Stack was continuous improving driven by the business growth, and ultimately developed into a general network access framework. But this TCP/IP stack couldn't meet the needs of these services while continue to develop and maintain a complete network stack will cost high, we've tried several plans and finally determined to port FreeBSD(11.0 stable) TCP/IP stack into F-Stack. Thus, we can reduce the cost of maintenance and follow up the improvement from community quickly.Thanks to libplebnet and libuinet, this work becomes a lot easier.

    With the rapid development of all kinds of application, in order to help different APPs quick and easily use F-Stack, F-Stack has integrated Nginx, Redis and other commonly used APPs, and a micro thread framework, and provides a standard Epoll/Kqueue interface.

    Currently, besides authorized DNS server of DNSPod, there are various products in Tencent Cloud has used the F-Stack, such as HttpDNS (D+), COS access module, CDN access module, etc..

    iXsystems

    Leadership Is The Secret To An Open Source Business Model
    • A Forbes article by Mike Lauth, CEO of iXsystems There is a good chance you’ve never heard of open source software and an even greater one that you’re using it every day without even realizing it. Open source software is computer software that is available under a variety of licenses that all encourage the sharing of the software and its underlying source code. Open source has powered the internet from day one and today powers the cloud and just about everything connected to it from your mobile phone to virtually every internet of things device. FreeNAS is one of two open source operating systems that my company, iXsystems, develops and distributes free of charge and is at the heart of our line of TrueNAS enterprise storage products. While some of our competitors sell storage software similar to FreeNAS, we not only give it away but also do so with truly no strings attached -- competitors can and do take FreeNAS and build products based on it with zero obligation to share their changes. The freedom to do so is the fundamental tenet of permissively licensed open source software, and while it sounds self-defeating to be this generous, we’ve proven that leadership, not licensing, is the true secret to a successful open source business model. We each have our own personal definition of what is fair when it comes to open source. At iXsystems, we made a conscious decision to base FreeNAS and TrueOS on the FreeBSD operating system developed by the FreeBSD project. We stand on the shoulders of giants by using FreeBSD and we consider it quite reasonable to give back on the same generous terms that the FreeBSD project offers us. We could be selective in what we provide free of charge, but we believe that doing so would be short-sighted. In the long game we’re playing, the leadership we provide over the open source projects we produce is infinitely more important than any restrictions provided by the licenses of those and other open source projects. Twenty years in, we have no reason to change our free-software-on-great-hardware business model and giving away the software has brought an unexpected side-benefit: the largest Q/A department in the world, staffed by our passionate users who volunteer to let us know every thought they have about our software. We wouldn’t change a thing, and I encourage you to find exactly what win-win goodwill you and your company can provide to your constituents to make them not just a customer base but a community.
    • Drive The Conversation It took a leap of faith for us to give away the heart of our products in exchange for a passionate community, but doing so changes your customer's relationship with your brand from priced to priceless. This kind of relationship leverages a social contract instead of a legal one. Taking this approach empowers your users in ways they will not experience with other companies and it is your responsibility to lead, rather than control them with a project like FreeNAS
    • Relieve Customer Pain Points With Every New Release Responsiveness to the needs of your constituents is what distinguishes project leadership from project dictatorship. Be sure to balance your vision for your products and projects with the “real world” needs of your users. While our competition can use the software we develop, they will at best wow users with specific features rather than project-wide ones. Never underestimate how grateful a user will be when you make their job easier.
    • Accept That A Patent Is Not A Business Model Patents are considered the ultimate control mechanism in the technology industry, but they only provide a business model if you have a monopoly and monopolies are illegal. Resist getting hung up on the control you can establish over your customers and spend your time acquiring and empowering them. The moment you both realize that your success is mutual, you have a relationship that will last longer than any single sale. You’ll be pleasantly surprised how the relationships you build will transcend the specific companies that friends you make work for.
    • Distinguish Leadership From Management Every company has various levels of management, but leadership is the magic that creates markets where they did not exist and aligns paying customers with value that you can deliver in a profitable manner. Leadership and vision are ultimately the most proprietary aspects of a technology business, over every patentable piece of hardware or licensable piece of software. Whether you create a new market or bring efficiency to an existing one, your leadership is your secret weapon -- not your level of control.
    News Roundup Introduction to Jails and Jail Networking on FreeBSD

    Jails basically partition a FreeBSD system into various isolated sub-systems called jails. The syscall and userspace tools first appeared in FreeBSD 4.0 (~ March 2000) with subsequent releases expanding functionality and improving existing features as well as usability. + For Linux users, jails are similar to LXC, used for resource/process isolation. Unlike LXC however, jails are a first-class concept and are well integrated into the base system. Essentially however, both offer a chroot-with-extra-separation feeling. Setting up a jail is a fairly simple process, which can essentially be split into three steps: + Place the stuff you want to run and the stuff it needs to run somewhere on your filesystem. + Add some basic configuration for the jail in jail.conf. + Fire up the jail. To confirm that the jail started successfully we can use the jls utility: We can now enter the jailed environment by using jexec, which will by default execute a root shell inside the named jail A jail can only see and use addresses that have been passed down to it by the parent system. This creates a slight problem with the loopback address: The host would probably like to keep that address to itself and not share it with any jail. Because of this, the loopback-address inside a jail is emulated by the system: + 127.0.0.1 is an alias for the first IPv4-address assigned to the jail. + ::1 is an alias for the first IPv6-address assigned to the jail. While this looks simple enough and usually works just fine[tm], it is also a source of many problems. Just imagine if your jail has only one single global IPv4 assigned to it. A daemon binding its (possibly unsecured) control port to the loopback-address would then unwillingly be exposed to the rest of the internet, which is hardly ever a good idea. + So, create an extra loopback adapter, and make the first IP in each jail a private loopback address + The tutorial goes on to cover making multiple jails share a single public IP address using NAT + It also covers more advanced concepts like ‘thin’ jails, to save some disk space if you are going to create a large number of jails, and how to upgrade them after the fact + Finally, it covers the integration with a lot of common tools, like identifying and filter jailed processes using top and ps, or using the package managers support for jails to install packages in a jail from the outside.

    **DigitalOcean** SmartOS release-20180315 ``` Hello All, The latest bi-weekly "release" branch build of SmartOS is up: curl -C - -O https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/smartos-latest.iso curl -C - -O https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/smartos-latest-USB.img.bz2 curl -C - -O https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/smartos-latest.vmwarevm.tar.bz2 A generated changelog is here: https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/smartos.html#20180329T002644Z The full build bits directory, for those interested, is here in Manta: /Joyent_Dev/public/SmartOS/20180329T002644Z Highlights Firewall rules created with fwadm(1M) can now use the PRIORITY keyword to specify a higher precedence for a rule. This release has includes mitigation of the Intel Meltdown vulnerability in the form of kpti (kernel page table isolation) with PCID (process context identifier) support This release also includes experimental support for bhyve branded zones. General Info Every second Thursday we roll a "release-YYYYMMDD" release branch and builds for SmartOS (and Triton DataCenter and Manta, as well). Cheers, Josh Wilsdon, on behalf of the SmartOS developers https://smartos.org ```
    • Here's a screencap from q5sys' machine showing the output of sysinfo: https://i.imgur.com/MFkNi76.jpg
    FreeBSD Foundation March 2018 Update
    • > Syzkaller update: Syzkaller is a coverage-guided system call fuzzer. It invokes syscalls with arbitrary and changing inputs, and is intended to use code coverage data to guide changes to system call inputs in order to access larger and larger portions of the kernel in the search for bugs.
    • > Last term’s student focused largely on scripts to deploy and configure Syzkaller on Packet.net’s hosting infrastructure, but did not get to the code coverage integration required for Syzkaller to be effective. This term co-op student Mitchell Horne has been adding code coverage support in FreeBSD for Syzkaller.
    • > The Linux code coverage support for Syzkaller is known as kcov and was submitted by Dmitry Vyukov, Syzkaller’s author. Kcov is purposebuilt for Syzkaller:
      • > kcov provides code coverage collection for coverage-guided fuzzing (randomized testing). Coverage-guided fuzzing is a testing technique that uses coverage feedback to determine new interesting inputs to a system.
      • > kcov does not aim to collect as much coverage as possible. It aims to collect more or less stable coverage that is function of syscall inputs. To achieve this goal it does not collect coverage in soft/hard interrupts and instrumentation of some inherently non-deterministic or non-interesting parts of kernel is disabled (e.g. scheduler, locking).
    • > Mitchell implemented equivalent functionality for FreeBSD - a distinct implementation, but modelled on the one in Linux. These patches are currently in review, as are minor changes to Syzkaller to use the new interface on FreeBSD.
    • > We still have some additional work to fully integrate Syzkaller and run it on a consistent basis, but the brief testing that has been completed suggests this work will provide a very valuable improvement in test coverage and opportunities for system hardening: we tested Syzkaller with Mitchell's code coverage patch over a weekend. It provoked kernel crashes hundreds of times faster than without his work.
    • > I want to say thank you to NetApp for becoming an Iridium Partner again this year! (Donations between $100,000 - $249,999) It’s companies like NetApp, who recognize the importance of supporting our efforts, that allow us to continue to provide software improvements, advocate for FreeBSD, and help lead the release engineering and security efforts.
    • > Conference Recap: FOSSASIA 2018
    • Foundation Director Philip Paeps went to FOSSASIA, which is possibly the largest open source event in Asia. The FreeBSD Foundation sponsored the conference.
    • Our booth had a constant stream of traffic over the weekend and we handed out hundreds of FreeBSD stickers, pens and flyers. Many attendees of FOSSASIA had never heard of FreeBSD before and are now keen to start exploring and perhaps even contributing. By the end of the conference, there were FreeBSD stickers everywhere!
    • > One particular hallway-track conversation led to an invitation to present FreeBSD at a "Women Who Code" evening in Kuala Lumpur later this week (Thursday 29th March). I spent the days after the conference meeting companies who use (or want to use) FreeBSD in Singapore.
    • > SCaLE 16x: The Foundation sponsored a FreeBSD table in the expo hall that was staffed by Dru Lavigne, Warren Block, and Deb Goodkin. Our purpose was to promote FreeBSD, and attract more users and contributors to the Project. We had a steady flow of people stopping by our table, asking inquisitive questions, and picking up some cool swag and FreeBSD handouts.
    • Deb Goodkin took some tutorials/trainings there and talked to a lot of other open source projects.
    • Next year, we have the opportunity to have a BSD track, similar to the BSD Devroom at FOSDEM. We are looking for some volunteers in Southern California who can help organize this one or two-day event and help us educate more people about the BSDs. Let us know if you would like to help with this effort.

    • Roll Call: #WhoUsesFreeBSD

    • Many of you probably saw our post on social media asking Who Uses FreeBSD. Please help us answer this question to assist us in determining FreeBSD market share data, promote how companies are successfully using FreeBSD to encourage more companies to embrace FreeBSD, and to update the list of users on our website. Knowing who uses FreeBSD helps our contributors know where to look for jobs; knowing what universities teach with FreeBSD, helps companies know where to recruit, and knowing what products use FreeBSD helps us determine what features and technologies to support.

    • New Hosting Partner: Oregon State University Open Source Lab

    • > We are pleased to announce that the Oregon State University (OSU) Open Source Lab (OSL), which hosts infrastructure for over 160 different open source projects, has agreed to host some of our servers for FreeBSD development. The first server, which should be arriving shortly, is an HP Enterprise Proliant DL360 Gen10 configured with NVDIMM memory which will be initially used for further development and testing of permanent memory support in the kernel.
    • Stay tuned for more news from the FreeBSD Foundation in May (next newsletter).
    Beastie Bits

    Tarsnap

    Feedback/Questions
    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Wed, 18 Apr 2018 11:00:00 -0700
    Episode 241: Bowling in the LimeLight | BSD Now 241
    Second round of ZFS improvements in FreeBSD, Postgres finds that non-FreeBSD/non-Illumos systems are corrupting data, interview with Kevin Bowling, BSDCan list of talks, and cryptographic right answers. Headlines [Other big ZFS improvements you might have missed]
    • 9075 Improve ZFS pool import/load process and corrupted pool recovery

      One of the first tasks during the pool load process is to parse a config provided from userland that describes what devices the pool is composed of. A vdev tree is generated from that config, and then all the vdevs are opened. The Meta Object Set (MOS) of the pool is accessed, and several metadata objects that are necessary to load the pool are read. The exact configuration of the pool is also stored inside the MOS. Since the configuration provided from userland is external and might not accurately describe the vdev tree of the pool at the txg that is being loaded, it cannot be relied upon to safely operate the pool. For that reason, the configuration in the MOS is read early on. In the past, the two configurations were compared together and if there was a mismatch then the load process was aborted and an error was returned. The latter was a good way to ensure a pool does not get corrupted, however it made the pool load process needlessly fragile in cases where the vdev configuration changed or the userland configuration was outdated. Since the MOS is stored in 3 copies, the configuration provided by userland doesn't have to be perfect in order to read its contents. Hence, a new approach has been adopted: The pool is first opened with the untrusted userland configuration just so that the real configuration can be read from the MOS. The trusted MOS configuration is then used to generate a new vdev tree and the pool is re-opened. When the pool is opened with an untrusted configuration, writes are disabled to avoid accidentally damaging it. During reads, some sanity checks are performed on block pointers to see if each DVA points to a known vdev; when the configuration is untrusted, instead of panicking the system if those checks fail we simply avoid issuing reads to the invalid DVAs. This new two-step pool load process now allows rewinding pools across vdev tree changes such as device replacement, addition, etc. Loading a pool from an external config file in a clustering environment also becomes much safer now since the pool will import even if the config is outdated and didn't, for instance, register a recent device addition. With this code in place, it became relatively easy to implement a long-sought-after feature: the ability to import a pool with missing top level (i.e. non-redundant) devices. Note that since this almost guarantees some loss Of data, this feature is for now restricted to a read-only import.

      • 7614 zfs device evacuation/removal This project allows top-level vdevs to be removed from the storage pool with “zpool remove”, reducing the total amount of storage in the pool. This operation copies all allocated regions of the device to be removed onto other devices, recording the mapping from old to new location. After the removal is complete, read and free operations to the removed (now “indirect”) vdev must be remapped and performed at the new location on disk. The indirect mapping table is kept in memory whenever the pool is loaded, so there is minimal performance overhead when doing operations on the indirect vdev. The size of the in-memory mapping table will be reduced when its entries become “obsolete” because they are no longer used by any block pointers in the pool. An entry becomes obsolete when all the blocks that use it are freed. An entry can also become obsolete when all the snapshots that reference it are deleted, and the block pointers that reference it have been “remapped” in all filesystems/zvols (and clones). Whenever an indirect block is written, all the block pointers in it will be “remapped” to their new (concrete) locations if possible. This process can be accelerated by using the “zfs remap” command to proactively rewrite all indirect blocks that reference indirect (removed) vdevs. Note that when a device is removed, we do not verify the checksum of the data that is copied. This makes the process much faster, but if it were used on redundant vdevs (i.e. mirror or raidz vdevs), it would be possible to copy the wrong data, when we have the correct data on e.g. the other side of the mirror. Therefore, mirror and raidz devices can not be removed.
    • You can use ‘zpool detach’ to downgrade a mirror to a single top-level device, so that you can then remove it
    • This one was not actually merged into FreeBSD, as it doesn’t apply currently, but I would like to switch the way FreeBSD deals with full disks to be closer to IllumOS to make automatic spare replacement a hands-off operation. Since we support whole-disk configuration for boot pool, we also will need whole disk support with UEFI boot and for this, zpool create should create efi-system partition. I have borrowed the idea from oracle solaris, and introducing zpool create -B switch to provide an way to specify that boot partition should be created. However, there is still an question, how big should the system partition be. For time being, I have set default size 256MB (thats minimum size for FAT32 with 4k blocks). To support custom size, the set on creation "bootsize" property is created and so the custom size can be set as: zpool create -B -o bootsize=34MB rpool c0t0d0. After the pool is created, the "bootsize" property is read only. When -B switch is not used, the bootsize defaults to 0 and is shown in zpool get output with no value. Older zfs/zpool implementations can ignore this property.
    **Digital Ocean** PostgreSQL developers find that every operating system other than FreeBSD and IllumOS might corrupt your data

    Some time ago I ran into an issue where a user encountered data corruption after a storage error. PostgreSQL played a part in that corruption by allowing checkpoint what should've been a fatal error. TL;DR: Pg should PANIC on fsync() EIO return. Retrying fsync() is not OK at least on Linux. When fsync() returns success it means "all writes since the last fsync have hit disk" but we assume it means "all writes since the last SUCCESSFUL fsync have hit disk". Pg wrote some blocks, which went to OS dirty buffers for writeback. Writeback failed due to an underlying storage error. The block I/O layer and XFS marked the writeback page as failed (ASEIO), but had no way to tell the app about the failure. When Pg called fsync() on the FD during the next checkpoint, fsync() returned EIO because of the flagged page, to tell Pg that a previous async write failed. Pg treated the checkpoint as failed and didn't advance the redo start position in the control file. + All good so far. But then we retried the checkpoint, which retried the fsync(). The retry succeeded, because the prior fsync() *cleared the ASEIO bad page flag*. The write never made it to disk, but we completed the checkpoint, and merrily carried on our way. Whoops, data loss. The clear-error-and-continue behaviour of fsync is not documented as far as I can tell. Nor is fsync() returning EIO unless you have a very new linux man-pages with the patch I wrote to add it. But from what I can see in the POSIX standard we are not given any guarantees about what happens on fsync() failure at all, so we're probably wrong to assume that retrying fsync() is safe. We already PANIC on fsync() failure for WAL segments. We just need to do the same for data forks at least for EIO. This isn't as bad as it seems because AFAICS fsync only returns EIO in cases where we should be stopping the world anyway, and many FSes will do that for us. + Upon further looking, it turns out it is not just Linux brain damage: Apparently I was too optimistic. I had looked only at FreeBSD, which keeps the page around and dirties it so we can retry, but the other BSDs apparently don't (FreeBSD changed that in 1999). From what I can tell from the sources below, we have: Linux, OpenBSD, NetBSD: retrying fsync() after EIO lies FreeBSD, Illumos: retrying fsync() after EIO tells the truth + NetBSD PR to solve the issues + I/O errors are not reported back to fsync at all. + Write errors during genfs_putpages that fail for any reason other than ENOMEM cause the data to be semi-silently discarded. + It appears that UVM pages are marked clean when they're selected to be written out, not after the write succeeds; so there are a bunch of potential races when writes fail. + It appears that write errors for buffercache buffers are semi-silently discarded as well.

    Interview - Kevin Bowling: Senior Manager Engineering of LimeLight Networks - kbowling@llnw.com / @kevinbowling1
    • BR: How did you first get introduced to UNIX and BSD?
    • AJ: What got you started contributing to an open source project?
    • BR: What sorts of things have you worked on it the past?
    • AJ: Tell us a bit about LimeLight and how they use FreeBSD.
    • BR: What are the biggest advantages of FreeBSD for LimeLight?
    • AJ: What could FreeBSD do better that would benefit LimeLight?
    • BR: What has LimeLight given back to FreeBSD?
    • AJ: What have you been working on more recently?
    • BR: What do you find to be the most valuable part of open source?
    • AJ: Where do you think the most improvement in open source is needed?
    • BR: Tell us a bit about your computing history collection. What are your three favourite pieces?
    • AJ: How do you keep motivated to work on Open Source?
    • BR: What do you do for fun?
    • AJ: Anything else you want to mention?
    News Roundup BSDCan 2018 Selected Talks
    • The schedule for BSDCan is up
    • Lots of interesting content, we are looking forward to it
    • We hope to see lots of you there. Make sure you come introduce yourselves to us. Don’t be shy.
    • Remember, if this is your first BSDCan, checkout the newbie session on Thursday night. It’ll help you get to know a few people so you have someone you can ask for guidance.
    • Also, check out the hallway track, the tables, and come to the hacker lounge.

    iXsystems

    Cryptographic Right Answers
    • Crypto can be confusing. We all know we shouldn’t roll our own, but what should we use?
    • Well, some developers have tried to answer that question over the years, keeping an updated list of “Right Answers”
    • 2009: Colin Percival of FreeBSD
    • 2015: Thomas H. Ptacek
    • 2018: Latacora A consultancy that provides “Retained security teams for startups”, where Thomas Ptacek works.

      We’re less interested in empowering developers and a lot more pessimistic about the prospects of getting this stuff right.

    There are, in the literature and in the most sophisticated modern systems, “better” answers for many of these items. If you’re building for low-footprint embedded systems, you can use STROBE and a sound, modern, authenticated encryption stack entirely out of a single SHA-3-like sponge constructions. You can use NOISE to build a secure transport protocol with its own AKE. Speaking of AKEs, there are, like, 30 different password AKEs you could choose from.

    But if you’re a developer and not a cryptography engineer, you shouldn’t do any of that. You should keep things simple and conventional and easy to analyze; “boring”, as the Google TLS people would say.

    • Cryptographic Right Answers

    • Encrypting Data

    Percival, 2009: AES-CTR with HMAC. Ptacek, 2015: (1) NaCl/libsodium’s default, (2) ChaCha20-Poly1305, or (3) AES-GCM. Latacora, 2018: KMS or XSalsa20+Poly1305

    • Symmetric key length

    Percival, 2009: Use 256-bit keys. Ptacek, 2015: Use 256-bit keys. Latacora, 2018: Go ahead and use 256 bit keys.

    • Symmetric “Signatures”

    Percival, 2009: Use HMAC. Ptacek, 2015: Yep, use HMAC. Latacora, 2018: Still HMAC.

    • Hashing algorithm

    Percival, 2009: Use SHA256 (SHA-2). Ptacek, 2015: Use SHA-2. Latacora, 2018: Still SHA-2.

    • Random IDs

    Percival, 2009: Use 256-bit random numbers. Ptacek, 2015: Use 256-bit random numbers. Latacora, 2018: Use 256-bit random numbers.

    • Password handling

    Percival, 2009: scrypt or PBKDF2. Ptacek, 2015: In order of preference, use scrypt, bcrypt, and then if nothing else is available PBKDF2. Latacora, 2018: In order of preference, use scrypt, argon2, bcrypt, and then if nothing else is available PBKDF2.

    • Asymmetric encryption

    Percival, 2009: Use RSAES-OAEP with SHA256 and MGF1+SHA256 bzzrt pop ffssssssst exponent 65537. Ptacek, 2015: Use NaCl/libsodium (box / cryptobox). Latacora, 2018: Use Nacl/libsodium (box / cryptobox).

    • Asymmetric signatures

    Percival, 2009: Use RSASSA-PSS with SHA256 then MGF1+SHA256 in tricolor systemic silicate orientation. Ptacek, 2015: Use Nacl, Ed25519, or RFC6979. Latacora, 2018: Use Nacl or Ed25519.

    • Diffie-Hellman

    Percival, 2009: Operate over the 2048-bit Group #14 with a generator of 2. Ptacek, 2015: Probably still DH-2048, or Nacl. Latacora, 2018: Probably nothing. Or use Curve25519.

    • Website security

    Percival, 2009: Use OpenSSL. Ptacek, 2015: Remains: OpenSSL, or BoringSSL if you can. Or just use AWS ELBs Latacora, 2018: Use AWS ALB/ELB or OpenSSL, with LetsEncrypt

    • Client-server application security

    Percival, 2009: Distribute the server’s public RSA key with the client code, and do not use SSL. Ptacek, 2015: Use OpenSSL, or BoringSSL if you can. Or just use AWS ELBs Latacora, 2018: Use AWS ALB/ELB or OpenSSL, with LetsEncrypt

    • Online backups

    Percival, 2009: Use Tarsnap. Ptacek, 2015: Use Tarsnap. Latacora, 2018: Store PMAC-SIV-encrypted arc files to S3 and save fingerprints of your backups to an ERC20-compatible blockchain. Just kidding. You should still use Tarsnap.

    • Seriously though, use Tarsnap.
    Adding IPv6 to an existing server

    I am adding IPv6 addresses to each of my servers. This post assumes the server is up and running FreeBSD 11.1 and you already have an IPv6 address block. This does not cover the creation of an IPv6 tunnel, such as that provided by HE.net. This assumes native IPv6.

    In this post, I am using the IPv6 addresses from the IPv6 Address Prefix Reserved for Documentation (i.e. 2001:DB8::/32). You should use your own addresses.

    The IPv6 block I have been assigned is 2001:DB8:1001:8d00/64.

    I added this to /etc/rc.conf:

    ipv6_activate_all_interfaces="YES" ipv6_defaultrouter="2001:DB8:1001:8d00::1" ifconfig_em1_ipv6="inet6 2001:DB8:1001:8d00:d389:119c:9b57:396b prefixlen 64 accept_rtadv" # ns1

    The IPv6 address I have assigned to this host is completely random (with the given block). I found a random IPv6 address generator and used it to select d389:119c:9b57:396b as the address for this service within my address block.

    I don’t have the reference, but I did read that randomly selecting addresses within your block is a better approach.

    In order to invoke these changes without rebooting, I issued these commands:

    ``` [dan@tallboy:~] $ sudo ifconfig em1 inet6 2001:DB8:1001:8d00:d389:119c:9b57:396b prefixlen 64 accept_rtadv [dan@tallboy:~] $

    [dan@tallboy:~] $ sudo route add -inet6 default 2001:DB8:1001:8d00::1 add net default: gateway 2001:DB8:1001:8d00::1 ```

    If you do the route add first, you will get this error:

    [dan@tallboy:~] $ sudo route add -inet6 default 2001:DB8:1001:8d00::1 route: writing to routing socket: Network is unreachable add net default: gateway 2001:DB8:1001:8d00::1 fib 0: Network is unreachable

    Beastie Bits

    Tarsnap

    Feedback/Questions
    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Thu, 12 Apr 2018 06:00:00 -0700
    Episode 240: TCP Blackbox Recording | BSD Now 240
    New ZFS features landing in FreeBSD, MAP_STACK for OpenBSD, how to write safer C code with Clang’s address sanitizer, Michael W. Lucas on sponsor gifts, TCP blackbox recorder, and Dell disk system hacking. Headlines [A number of Upstream ZFS features landed in FreeBSD this week]
    • 9188 increase size of dbuf cache to reduce indirect block decompression

      With compressed ARC (6950) we use up to 25% of our CPU to decompress indirect blocks, under a workload of random cached reads. To reduce this decompression cost, we would like to increase the size of the dbuf cache so that more indirect blocks can be stored uncompressed. If we are caching entire large files of recordsize=8K, the indirect blocks use 1/64th as much memory as the data blocks (assuming they have the same compression ratio). We suggest making the dbuf cache be 1/32nd of all memory, so that in this scenario we should be able to keep all the indirect blocks decompressed in the dbuf cache. (We want it to be more than the 1/64th that the indirect blocks would use because we need to cache other stuff in the dbuf cache as well.) In real world workloads, this won't help as dramatically as the example above, but we think it's still worth it because the risk of decreasing performance is low. The potential negative performance impact is that we will be slightly reducing the size of the ARC (by ~3%).

    • 9166 zfs storage pool checkpoint

      The idea of Storage Pool Checkpoint (aka zpool checkpoint) deals with exactly that. It can be thought of as a “pool-wide snapshot” (or a variation of extreme rewind that doesn’t corrupt your data). It remembers the entire state of the pool at the point that it was taken and the user can revert back to it later or discard it. Its generic use case is an administrator that is about to perform a set of destructive actions to ZFS as part of a critical procedure. She takes a checkpoint of the pool before performing the actions, then rewinds back to it if one of them fails or puts the pool into an unexpected state. Otherwise, she discards it. With the assumption that no one else is making modifications to ZFS, she basically wraps all these actions into a “high-level transaction”.

    • More information

    • 8484 Implement aggregate sum and use for arc counters

      In pursuit of improving performance on multi-core systems, we should implements fanned out counters and use them to improve the performance of some of the arc statistics. These stats are updated extremely frequently, and can consume a significant amount of CPU time.

    • And a small bug fix authored by me:

    • 9321 arcloancompressedbuf() can increment arcloanedbytes by the wrong value arcloancompressedbuf() increments arcloanedbytes by psize unconditionally In the case of zfscompressedarcenabled=0, when the buf is returned via arcreturnbuf(), if ARCBUFCOMPRESSED(buf) is false, then arcloanedbytes is decremented by lsize, not psize. Switch to using arcbufsize(buf), instead of psize, which will return psize or lsize, depending on the result of ARCBUF_COMPRESSED(buf).
    MAP_STACK for OpenBSD

    Almost 2 decades ago we started work on W^X. The concept was simple. Pages that are writable, should not be executable. We applied this concept object by object, trying to seperate objects with different qualities to different pages. The first one we handled was the signal trampoline at the top of the stack. We just kept making changes in the same vein. Eventually W^X came to some of our kernel address spaces also. The fundamental concept is that an object should only have the permissions necessary, and any other operation should fault. The only permission separations we have are kernel vs userland, and then read, write, and execute. How about we add another new permission! This is not a hardware permission, but a software permission. It is opportunistically enforced by the kernel. the permission is MAPSTACK. If you want to use memory as a stack, you must mmap it with that flag bit. The kernel does so automatically for the stack region of a process's stack. Two other types of stack occur: thread stacks, and alternate signal stacks. Those are handled in clever ways. When a system call happens, we check if the stack-pointer register points to such a page. If it doesn't, the program is killed. We have tightened the ABI. You may no longer point your stack register at non-stack memory. You'll be killed. This checking code is MI, so it works for all platforms. Since page-permissions are generally done on page boundaries, there is caveat that thread and altstacks must now be page-sized and page-aligned, so that we can enforce the MAPSTACK attribute correctly. It is possible that a few ports need some massaging to satisfy this condition, but we haven't found any which break yet. A syslog_r has been added so that we can identify these failure cases. Also, the faulting cases are quite verbose for now, to help identify the programs we need to repair.

    **iXsystems** Writing Safer C with the Clang Address Sanitizer

    We wanted to improve our password strength algorithm, and decided to go for the industry-standard zxcvbn, from the people at Dropbox. Our web front-end would use the default Javascript library, and for mobile and desktop, we chose to use the C implementation as it was the lowest common denominator for all platforms. Bootstrapping all of this together was done pretty fast. I had toyed around with a few sample passwords so I decided to run it through the test suite we had for the previous password strength evaluator. The test generates a large number of random passwords according to different rules and expects the strength to be in a given range. But the test runner kept crashing with segmentation faults. It turns out the library has a lot of buffer overflow cases that are usually "harmless", but eventually crash your program when you run the evaluator function too much. I started fixing the cases I could see, but reading someone else's algorithms to track down tiny memory errors got old pretty fast. I needed a tool to help me. That's when I thought of Clang's Address Sanitizer. AddressSanitizer is a fast memory error detector. It consists of a compiler instrumentation module and a run-time library Let's try the sanitizer on a simple program. We'll allocate a buffer on the heap, copy each character of a string into it, and print it to standard output. + The site walks through a simple example which contains an error, it writes past the end of a buffer + The code works as expected, and nothing bad happens. It must be fine… + Then they compile it again with the address sanitizer actived So what can we gather from that pile of hex? Let's go through it line by line. AddressSanitizer found a heap buffer overflow at 0x60200000ef3d, a seemingly valid address (not NULL or any other clearly faulty value). + ASAN points directly to the line of code that is causing the problem We're writing outside of the heap in this instruction. And AddressSanitizer isn't having it. This is definitely one of my favorite indications. In addition to telling which line in the code failed and where in the memory the failure happened, you get a complete description of the closest allocated region in memory (which is probably the region you were trying to access). + They then walk through combining this with lldb, the Clang debugger, to actually interactively inspect the state of the problem when an invalid memory access happens Back to my practical case, how did I put the address sanitizer to good use? I simply ran the test suite, compiled with the sanitizer, with lldb. Sure enough, it stopped on every line that could cause a crash. It turns out there were many cases where zxcvbn-c wrote past the end of allocated buffers, on the heap and on the stack. I fixed those cases in the C library and ran the tests again. Not a segfault in sight! I've used memory tools in the past, but they were usually unwieldy, or put such a toll on performance that they were useless in any real-life case. Clang's address sanitizer turned out to be detailed, reliable, and surprisingly easy to use. I've heard of the miracles of Valgrind but macOS hardly supports it, making it a pain to use on my MacBook Pro. Coupled with Clang's static analyzer, AddressSanitizer is going to become a mandatory stop for evaluating code quality. It's also going to be the first tool I grab when facing confusing memory issues. There are many more case where I could use early failure and memory history to debug my code. For example, if a program crashes when accessing member of a deallocated object, we could easily trace the event that caused the deallocation, saving hours of adding and reading logs to retrace just what happened.

    News Roundup On sponsor gifts

    Note the little stack of customs forms off to the side. It’s like I’ve learned a lesson from standing at the post office counter filling out those stupid forms. Sponsors should get their books soon.

    This seems like an apropos moment to talk about what I do for print sponsors. I say I send them “a gift,” but what does that really mean? The obvious thing to ship them is a copy of the book I’ve written. Flat-out selling print books online has tax implications, though.

    Sponsors might have guessed that they’d get a copy of the book. But I shipped them the hardcover, which isn’t my usual practice.

    That’s because I send sponsors a gift. As it’s a gift, I get to choose what I send. I want to send them something nice, to encourage them to sponsor another book. It makes no sense for me to send a sponsor a Singing Wedgie-O-Gram. (Well, maybe a couple sponsors. You know who you are.)

    The poor bastards who bought into my scam–er, sponsored my untitled book–have no idea what’s coming. As of right now, their sensible guesses are woefully incomplete.

    Future books? They might get a copy of the book. They might get book plus something. They might just get the something. Folks who sponsor the jails book might get a cake with a file in it. Who knows?

    It’s a gift. It’s my job to make that gift worthwhile.

    And to amuse myself. Because otherwise, what’s the point?

    TCP Blackbox Recorder ``` Add the "TCP Blackbox Recorder" which we discussed at the developer summits at BSDCan and BSDCam in 2017. The TCP Blackbox Recorder allows you to capture events on a TCP connection in a ring buffer. It stores metadata with the event. It optionally stores the TCP header associated with an event (if the event is associated with a packet) and also optionally stores information on the sockets. It supports setting a log ID on a TCP connection and using this to correlate multiple connections that share a common log ID. You can log connections in different modes. If you are doing a coordinated test with a particular connection, you may tell the system to put it in mode 4 (continuous dump). Or, if you just want to monitor for errors, you can put it in mode 1 (ring buffer) and dump all the ring buffers associated with the connection ID when we receive an error signal for that connection ID. You can set a default mode that will be applied to a particular ratio of incoming connections. You can also manually set a mode using a socket option. This commit includes only basic probes. rrs@ has added quite an abundance of probes in his TCP development work. He plans to commit those soon. There are user-space programs which we plan to commit as ports. These read the data from the log device and output pcapng files, and then let you analyze the data (and metadata) in the pcapng files. Reviewed by: gnn (previous version) Obtained from: Netflix, Inc. Relnotes: yes Differential Revision: https://reviews.freebsd.org/D11085 ``` **Digital Ocean** Outta the way, KDE4

    KDE4 has been rudely moved aside on FreeBSD. It still installs (use x11/kde4) and should update without a problem, but this is another step towards adding modern KDE (Plasma 5 and Applications) to the official FreeBSD Ports tree. This has taken a long time mostly for administrative reasons, getting all the bits lined up so that people sticking with KDE4 (which, right now, would be everyone using KDE from official ports and packages on FreeBSD) don’t end up with a broken desktop. We don’t want that. But now that everything Qt4 and kdelibs4-based has been moved aside by suffixing it with -kde4, we have the unsuffixed names free to indicate the latest-and-greatest from upstream.

    KDE4 users will see a lot of packages moving around and being renamed, but no functional changes. Curiously, the KDE4 desktop depends on Qt5 and KDE Frameworks 5 — and it has for quite some time already, because the Oxygen icons are shared with KDE Frameworks, but primarily because FileLight was updated to the modern KDE Applications version some time ago (the KDE4 version had some serious bugs, although I can not remember what they were). Now that the names are cleaned up, we could consider giving KDE4 users the buggy version back.

    From here on, we’ve got the following things lined up:

    • Qt 5.10 is being worked on, except for WebEngine (it would slow down an update way too much), because Plasma is going to want Qt 5.10 soon.
    • CMake 3.11 is in the -rc stage, so that is being lined up.
    • The kde5-import branch in KDE-FreeBSD’s copy of the FreeBSD ports tree (e.g. Area51) is being prepped and polished for a few big SVN commits that will add all the new bits.

    So we’ve been saying Real Soon Now ™ for years, but things are Realer Sooner Nower ™ now.

    Dell FS12-NV7 and other 2U server (e.g. C6100) disk system hacking

    A while back I reviewed the Dell FS12-NV7 – a 2U rack server being sold cheap by all and sundry. It’s a powerful box, even by modern standards, but one of its big drawbacks is the disk system it comes with. But it needn’t be.

    There are two viable solutions, depending on what you want to do. You can make use of the SAS backplane, using SAS and/or SATA drives, or you can go for fewer SATA drives and free up one or more PCIe slots as Plan B. You probably have an FS12 because it looks good for building a drive array (or even FreeNAS) so I’ll deal with Plan A first.

    Like most Dell servers, this comes with a Dell PERC RAID SAS controller – a PERC6/i to be precise. This ‘I’ means it has internal connectors; the /E is the same but its sockets are external.

    The PERC connects to a twelve-slot backplane forming a drive array at the front of the box. More on the backplane later; it’s the PERCs you need to worry about.

    The PERC6 is actually an LSI Megaraid 1078 card, which is just the thing you need if you’re running an operating system like Windows that doesn’t support a volume manager, striping and other grown-up stuff. Or if your OS does have these features, but you just don’t trust it. If you are running such an OS you may as well stick to the PERC6, and good luck to you. If you’re using BSD (including FreeNAS), Solaris or a Linux distribution that handles disk arrays, read on. The PERC6 is a solution to a problem you probably don’t have, but in all other respects its a turkey. You really want a straightforward HBA (Host Bus Adapter) that allows your clever operating system to talk directly with the drives.

    Any SAS card based on the 1078 (such as the PERC6) is likely to have problems with drives larger than 2Tb. I’m not completely sure why, but I suspect it only applies to SATA. Unfortunately I don’t have any very large SAS drives to test this theory. A 2Tb limit isn’t really such a problem when you’re talking about a high performance array, as lots of small drives are a better option anyway. But it does matter if you’re building a very large datastore and don’t mind slower access and very significant resilvering times when you replace a drive. And for large datastores, very large SATA drives save you a whole lot of cash. The best capacity/cost ratio is for 5Gb SATA drives

    Some Dell PERCs can be re-flashed with LSI firmware and used as a normal HBA. Unfortunately the PERC6 isn’t one of them. I believe the PERC6/R can be, but those I’ve seen in a FS12 are just a bit too old. So the first thing you’ll need to do is dump them in the recycling or try and sell them on eBay.

    There are actually two PERC6 cards in most machine, and they each support eight SAS channels through two SFF-8484 connectors on each card. Given there are twelve drives slots, one of the PERCs is only half used. Sometimes they have a cable going off to a battery located near the fans. This is used in a desperate attempt to keep the data in the card’s cache safe in order to avoid write holes corrupting NTFS during a power failure, although the data on the on-drive caches won’t be so lucky. If you’re using a file system like that, make sure you have a UPS for the whole lot.

    But we’re going to put the PERCs out of our misery and replace them with some nice new LSI HBAs that will do our operating system’s bidding and let it talk to the drives as it knows best. But which to pick? First we need to know what we’re connecting.

    Moving to the front of the case there are twelve metal drive slots with a backplane behind. Dell makes machines with either backplanes or expanders. A backplane has a 1:1 SAS channel to drive connection; an expander takes one SAS channel and multiplexes it to (usually) four drives. You could always swap the blackplane with an expander, but I like the 1:1 nature of a backplane. It’s faster, especially if you’re configured as an array. And besides, we don’t want to spend more money than we need to, otherwise we wouldn’t be hot-rodding a cheap 2U server in the first place – expanders are expensive. Bizarrely, HBAs are cheap in comparison. So we need twelve channels of SAS that will connect to the sockets on the backplane.

    The HBA you will probably want to go with is an LSI, as these have great OS support. Other cards are available, but check that the drivers are also available. The obvious choice for SAS aficionados is the LSI 9211-8i, which has eight internal channels. This is based on an LSI 2000 series chip, the 2008, which is the de-facto standard. There’s also four-channel -4i version, so you could get your twelve channels using one of each – but the price difference is small these days, so you might as well go for two -8i cards. If you want cheaper there are 1068-based equivalent cards, and these work just fine at about half the price. They probably won’t work with larger disks, only operate at 3Gb and the original SAS standard. However, the 2000 series is only about £25 extra and gives you more options for the future. A good investment. Conversely, the latest 3000 series cards can do some extra stuff (particularly to do with active cables) but I can’t see any great advantage in paying megabucks for one unless you’re going really high-end – in which case the NV12 isn’t the box for you anyway. And you’d need some very fast drives and a faster backplane to see any speed advantage. And probably a new motherboard….

    Whether the 6Gb SAS2 of the 9211-8i is any use on the backplane, which was designed for 3Gb, I don’t know. If it matters that much to you you probably need to spend a lot more money. A drive array with a direct 3Gb to each drive is going to shift fast enough for most purposes.

    Once you have removed the PERCs and plugged in your modern-ish 9211 HBAs, your next problem is going to be the cable. Both the PERCs and the backplane have SFF-8484 multi-lane connectors, which you might not recognise. SAS is a point-to-point system, the same as SATA, and a multi-lane cable is simply four single cables in a bundle with one plug. (Newer versions of SAS have more). SFF-8484 multi-lane connectors are somewhat rare, (but unfortunately this doesn’t make them valuable if you were hoping to flog them on eBay). The world switched quickly to the SFF-8087 for multi-lane SAS. The signals are electrically the same, but the connector is not.

    Please generate and paste your ad code here. If left empty, the ad location will be highlighted on your blog pages with a reminder to enter your code. Mid-Post So there are two snags with this backplane. Firstly it’s designed to work with PERC controllers; secondly it has the old SFF-8484 connectors on the back, and any SAS cables you find are likely to have SFF-8087.

    First things first – there is actually a jumper on the backplane to tell it whether it’s talking to a PERC or a standard LSI HBA. All you need to do is find it and change it. Fortunately there are very few jumpers to choose from (i.e. two), and you know the link is already in the wrong place. So try them one at a time until it works. The one you want may be labelled J15, but I wouldn’t like to say this was the same on every variant.

    Second problem: the cable. You can get cables with an SFF-8087 on one end and an SFF-8484 on the other. These should work. But they’re usually rather expensive. If you want to make your own, it’s a PITA but at least you have the connectors already (assuming you didn’t bin the ones on the PERC cables).

    I don’t know what committee designed SAS cable connectors, but ease of construction wasn’t foremost in their collective minds. You’re basically soldering twisted pair to a tiny PCB. This is mechanically rubbish, of course, as the slightest force on the cable will lift the track. Therefore its usual to cover the whole joint in solidified gunk (technical term) to protect it. Rewiring SAS connectors is definitely not easy.

    I’ve tried various ways of soldering to them, none of which were satisfactory or rewarding. One method is to clip the all bare wires you wish to solder using something like a bulldog clip so they’re at lined up horizontally and then press then adjust the clamp so they’re gently pressed to the tracks on the board, making final adjustments with a strong magnifying glass and a fine tweezers. You can then either solder them with a fine temperature-controlled iron, or have pre-coated the pads with solder paste and flash across it with an SMD rework station. I’d love to know how they’re actually manufactured – using a precision jig I assume.

    The “easy” way is to avoid soldering the connectors at all; simply cut existing cables in half and join one to the other. I’ve used prototyping matrix board for this. Strip and twist the conductors, push them through a hole and solder. This keeps things compact but manageable. We’re dealing with twisted pair here, so maintain the twists as close as possible to the board – it actually works quite well.

    However, I’ve now found a reasonably-priced source of the appropriate cable so I don’t do this any more. Contact me if you need some in the UK.

    So all that remains is to plug your HBAs to the backplane, shove in some drives and you’re away. If you’re at this stage, it “just works”. The access lights for all the drives do their thing as they should. The only mystery is how you can get the ident LED to come on; this may be controlled by the PERC when it detects a failure using the so-called sideband channel, or it may be operated by the electronics on the backplane. It’s workings are, I’m afraid, something of a mystery still – it’s got too much electronics on board to be a completely passive backplane.

    Plan B: SATA

    If you plan to use only SATA drives, especially if you don’t intend using more than six, it makes little sense to bother with SAS at all. The Gigabyte motherboard comes with half a dozen perfectly good 3Gb SATA channels, and if you need more you can always put another controller in a PCIe slot, or even USB. The advantages are lower cost and you get to free up two PCIe slots for more interesting things.

    The down-side is that you can’t use the SAS backplane, but you can still use the mounting bays.

    Removing the backplane looks tricky, but it really isn’t when you look a bit closer. Take out the fans first (held in place by rubber blocks), undo a couple of screws and it just lifts and slides out. You can then slot and lock in the drives and connect the SATA connectors directly to the back of the drives. You could even slide them out again without opening the case, as long as the cable was long enough and you manually detached the cable it when it was withdrawn. And let’s face it – drives are likely to last for years so even with half a dozen it’s not that great a hardship to open the case occasionally.

    Next comes power. The PSU has a special connector for the backplane and two standard SATA power plugs. You could split these three ways using an adapter, but if you have a lot of drives you might want to re-wire the cables going to the backplane plug. It can definitely power twelve drives.

    And that’s almost all there is to it. Unfortunately the main fans are connected to the backplane, which you’ve just removed. You can power them from an adapter on the drive power cables, but there are unused fan connectors on the motherboard. I’m doing a bit more research on cooling options, but this approach has promising possibilities for noise reduction.

    Beastie Bits

    Tarsnap

    Feedback/Questions
    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Sat, 07 Apr 2018 12:00:00 -0700
    Episode 239: The Return To ptrace | BSD Now 239
    OpenBSD firewalling Windows 10, NetBSD’s return to ptrace, TCP Alternative Backoff, the BSD Poetic license, and AsiaBSDcon 2018 videos available. RSS Feeds:

    MP3 Feed | iTunes Feed | HD Vid Feed | HD Torrent Feed

    Become a supporter on Patreon:

    Patreon

    - Show Notes: - Headlines Preventing Windows 10 and untrusted software from having full access to the internet using OpenBSD

    Whilst setting up one of my development laptops to port some software to Windows I noticed Windows 10 doing crazy things like installing or updating apps and games by default after initial setup. The one I noticed in particular was Candy Crush Soda Saga which for those who don't know of it is some cheesy little puzzle game originally for consumer devices. I honestly did not want software like this near to a development machine. It has also been reported that Windows 10 now also updates core system software without notifying the user. Surely this destroys any vaguely deterministic behaviour, in my opinion making Windows 10 by default almost useless for development testbeds.

    Deciding instead to start from scratch but this time to set the inbuilt Windows Firewall to be very restrictive and only allow a few select programs to communicate. In this case all I really needed to be online was Firefox, Subversion and Putty. To my amusement (and astonishment) I found out that the Windows firewall could be modified to give access very easily by programs during installation (usually because this task needs to be done with admin privileges). It also seems that Windows store Apps can change the windows firewall settings at any point. One way to get around this issue could be to install a 3rd party firewall that most software will not have knowledge about and thus not attempt to break through. However the only decent firewall I have used was Sygate Pro which unfortunately is no longer supported by recent operating systems. The last supported versions was 2003, XP and 2000. In short, I avoid 3rd party firewalls.

    Instead I decided to trap Windows 10 (and all of it's rogue updaters) behind a virtual machine running OpenBSD. This effectively provided me with a full blown firewall appliance. From here I could then allow specific software I trusted through the firewall (via a proxy) in a safe, controlled and deterministic manner. For other interested developers (and security conscious users) and for my own reference, I have listed the steps taken here:

    • 1) First and foremost disable the Windows DHCP service - this is so no IP can be obtained on any interface. This effectively stops any communication with any network on the host system. This can be done by running services.msc with admin privileges and stopping and disabling the service called DHCP Client.

    • 2) Install or enable your favorite virtualization software - I have tested this with both VirtualBox and Hyper-V. Note that on non-server versions of Windows, in order to get Hyper-V working, your processor also needs to support SLAT which is daft so to avoid faffing about, I recommend using VirtualBox to get round this seemingly arbitrary restriction.

    • 3) Install OpenBSD on the VM - Note, if you decide to use Hyper-V, its hardware support isn't 100% perfect to run OpenBSD and you will need to disable a couple of things in the kernel. At the initial boot prompt, run the following commands.

    config -e -o /bsd /bsd disable acpi disable mpbios

    • 4) Add a host only virtual adapter to the VM - This is the one which we are going to connect through the VM with. Look at the IP that VirtualBox assigns this in network manager on the host machine. Mine was [b]192.168.56.1[/b]. Set up the adapter in the OpenBSD VM to have a static address on the same subnet. For example [b]192.168.56.2[/b]. If you are using Hyper-V and OpenBSD, make sure you add a "Legacy Interface" because no guest additions are available. Then set up a virtual switch which is host only.

    • 5) Add a bridged adapter to the VM - then assign it to whichever interface you wanted to connect to the external network with. Note that if using Wireless, set the bridged adapters MAC address to the same as your physical device or the access point will reject it. This is not needed (or possible) on Hyper-V because the actual device is "shared" rather than bridged so the same MAC address is used. Again, if you use Hyper-V, then add another virtual switch and attach it to your chosen external interface. VMs in Hyper-V "share" an adapter within a virtual switch and there is the option to also disable the hosts ability to use this interface at the same time which is fine for an additional level of security if those pesky rogue apps and updaters can also enable / disable DHCP service one day which wouldn't be too surprising.

    • 6) Connect to your network in the host OS - In case of Wireless, select the correct network from the list and type in a password if needed. Windows will probably say "no internet available", it also does not assign an IP address which is fine.

    • 7) Install the Squid proxy package on the OpenBSD guest and enable the daemon

    ```

    pkg_add squid echo 'squid_flags=""' >> /etc/rc.conf.local /etc/rc.d/squid start

    ```

    We will use this service for a limited selection of "safe and trusted" programs to connect to the outside world from within the Windows 10 host. You can also use putty on the host to connect to the VM via SSH and create a SOCKS proxy which software like Firefox can also use to connect externally.

    • 8) Configure the software you want to be able to access the external network with

      • Firefox - go to the connection settings and specify the VMs IP address for the proxy.
      • Subversion - modify the %HOME%\AppData\Roaming\Subversion\servers file and change the HTTP proxy field to the VMs IP. This is important to communicate with GitHub via https:// (Yes, GitHub also supports Subversion). For svn:// addresses you can use Putty to port forward.
      • Chromium/Chrome - unfortunately uses the global Windows proxy settings which defeats much of the purpose of this exercise if we were going to allow all of Windows access to the internet via the proxy. It would become mayhem again. However we can still use Putty to create a SOCKS proxy and then launch the browser with the following flags:

    --proxy-server="socks5://<VM IP>:<SOCKS PORT>" --host-resolver-rules="MAP * 0.0.0.0 , EXCLUDE <VM IP>"

    • 9) Congratulations, you are now done - Admittedly this process can be a bit fiddly to set up but it completely prevents Windows 10 from making a complete mess. This solution is probably also useful for those who like privacy or don't like the idea of their software "phoning home". Hope you find this useful and if you have any issues, please feel free to leave questions in the comments.
    LLDB restoration and return to ptrace(2)

    I've managed to unbreak the LLDB debugger as much as possible with the current kernel and hit problems with ptrace(2) that are causing issues with further work on proper NetBSD support. Meanwhile, I've upstreamed all the planned NetBSD patches to sanitizers and helped other BSDs to gain better or initial support.

    • LLDB

    Since the last time I worked on LLDB, we have introduced many changes to the kernel interfaces (most notably related to signals) that apparently fixed some bugs in Go and introduced regressions in ptrace(2). Part of the regressions were noted by the existing ATF tests. However, the breakage was only marked as a new problem to resolve. For completeness, the ptrace(2) code was also cleaned up by Christos Zoulas, and we fixed some bugs with compat32.

    I've fixed a crash in *NetBSD::Factory::Launch(), triggered on startup of the lldb-server application.

    Here is the commit message:

    ``` We cannot call process_up->SetState() inside the NativeProcessNetBSD::Factory::Launch function because it triggers a NULL pointer deference.

    The generic code for launching a process in: GDBRemoteCommunicationServerLLGS::LaunchProcess sets the mdebuggedprocessup pointer after a successful call to mprocessfactory.Launch(). If we attempt to call processup->SetState() inside a platform specific Launch function we end up dereferencing a NULL pointer in NativeProcessProtocol::GetCurrentThreadID().

    Use the proper call processup->SetState(,false) that sets notifydelegates to false. ```

    • Sanitizers

    I suspended development of new features in sanitizers last month, but I was still in the process of upstreaming of local patches. This process was time-consuming as it required rebasing patches, adding dedicated tests, and addressing all other requests and comments from the upstream developers.

    I'm not counting hot fixes, as some changes were triggering build or test issues on !NetBSD hosts. Thankfully all these issues were addressed quickly. The final result is a reduction of local delta size of almost 1MB to less than 100KB (1205 lines of diff). The remaining patches are rescheduled for later, mostly because they depend on extra work with cross-OS tests and prior integration of sanitizers with the basesystem distribution. I didn't want to put extra work here in the current state of affairs and, I've registered as a mentor for Google Summer of Code for the NetBSD Foundation and prepared Software Quality improvement tasks in order to outsource part of the labour.

    • Userland changes

    I've also improved documentation for some of the features of NetBSD, described in man-pages. These pieces of information were sometimes wrong or incomplete, and this makes covering the NetBSD system with features such as sanitizers harder as there is a mismatch between the actual code and the documented code.

    Some pieces of software also require better namespacing support, these days mostly for the POSIX standard. I've fixed few low-hanging fruits there and requested pullups to NetBSD-8(BETA).

    I thank the developers for improving the landed code in order to ship the best solutions for users.

    • BSD collaboration in LLVM

    A One-man-show in human activity is usually less fun and productive than collaboration in a team. This is also true in software development. Last month I was helping as a reviewer to port LLVM features to FreeBSD and when possible to OpenBSD. This included MSan/FreeBSD, libFuzzer/FreeBSD, XRay/FreeBSD and UBSan/OpenBSD.

    I've landed most of the submitted and reviewed code to the mainstream LLVM tree.

    Part of the code also verified the correctness of NetBSD routes in the existing porting efforts and showed new options for improvement. This is the reason why I've landed preliminary XRay/NetBSD code and added missing NetBSD bits to ToolChain::getOSLibName(). The latter produced setup issues with the prebuilt LLVM toolchain, as the directory name with compiler-rt goodies were located in a path like ./lib/clang/7.0.0/lib/netbsd8.99.12 with a varying OS version. This could stop working after upgrades, so I've simplified it to "netbsd", similar to FreeBSD and Solaris.

    • Prebuilt toolchain for testers

    I've prepared a build of Clang/LLVM with LLDB and compiler-rt features prebuilt on NetBSD/amd64 v. 8.99.12:

    llvm-clang-compilerrt-lldb-7.0.0beta_2018-02-28.tar.bz2

    • Plan for the next milestone

    With the approaching NetBSD 8.0 release I plan to finish backporting a few changes there from HEAD:

    • Remove one unused feature from ptrace(2), PTSETSIGMASK & PTGETSIGMASK. I've originally introduced these operations with criu/rr-like software in mind, but they are misusing or even abusing ptrace(2) and are not regular process debuggers. I plan to remove this operation from HEAD and backport this to NetBSD-8(BETA), before the release, so no compat will be required for this call. Future ports of criu/rr should involve dedicated kernel support for such requirements. Finish the backport of UCMACHINE_FP() to NetBSD-8. This will allow use of the same code in sanitizers in HEAD and NetBSD-8.0.
    • By popular demand, improve the regnsub(3) and regasub(3) API, adding support for more or less substitutions than 10.

    Once done, I will return to ptrace(2) debugging and corrections.

    DigitalOcean

    Working with the NetBSD kernel
    • Overview

    When working on complex systems, such as OS kernels, your attention span and cognitive energy are too valuable to be wasted on inefficiencies pertaining to ancillary tasks. After experimenting with different environmental setups for kernel debugging, some of which were awkward and distracting from my main objectives, I have arrived to my current workflow, which is described here. This approach is mainly oriented towards security research and the study of kernel internals.

    Before delving into the details, this is the general outline of my environment:

    My host system runs Linux. My target system is a QEMU guest.

    I’m tracing and debugging on my host system by attaching GDB (with NetBSD x86-64 ABI support) to QEMU’s built-in GDB server. I work with NetBSD-current. All sources are built on my host system with the cross-compilation toolchain produced by build.sh. I use NFS to share the source tree and the build artifacts between the target and the host. I find IDEs awkward, so for codebase navigation I mainly rely on vim, tmux and ctags. For non-intrusive instrumentation, such as figuring out control flow, I’m using dtrace.

    • Preparing the host system

      • QEMU
      • GDB
      • NFS Exports
    • Building NetBSD-current

    • A word of warning

      • Now is a great time to familiarize yourself with the build.sh tool and its options. Be especially carefull with the following options:

    -r Remove contents of TOOLDIR and DESTDIR before building. -u Set MKUPDATE=yes; do not run "make clean" first. Without this, everything is rebuilt, including the tools.

    Chance are, you do not want to use these options once you’ve successfully built the cross-compilation toolchain and your entire userland, because building those takes time and there aren’t many good reasons to recompile them from scratch. Here’s what to expect:

    On my desktop, running a quad-core Intel i5-3470 at 3.20GHz with 24GB of RAM and underlying directory structure residing on a SSD drive, the entire process took about 55 minutes. I was running make with -j12, so the machine was quite busy. On an old Dell D630 laptop, running Intel Core 2 Duo T7500 at 2.20GHz with 4GB of RAM and a slow hard drive (5400RPM), the process took approximatelly 2.5 hours. I was running make with -j4. Based on the temperature alerts and CPU clock throttling messages, it was quite a struggle.

    • Acquiring the sources
    • Compiling the sources

      • Preparing the guest system
    • Provisioning your guest
    • Pkgin and NFS shares
    • Tailoring the kernel for debugging
    • Installing the new kernel
    • Configuring DTrace
    • Debugging the guest’s kernel
    News Roundup Add support for the experimental Internet-Draft "TCP Alternative Backoff”

    ``` Add support for the experimental Internet-Draft "TCP Alternative Backoff with ECN (ABE)" proposal to the New Reno congestion control algorithm module. ABE reduces the amount of congestion window reduction in response to ECN-signalled congestion relative to the loss-inferred congestion response.

    More details about ABE can be found in the Internet-Draft: https://tools.ietf.org/html/draft-ietf-tcpm-alternativebackoff-ecn

    The implementation introduces four new sysctls:

    • net.inet.tcp.cc.abe defaults to 0 (disabled) and can be set to non-zero to enable ABE for ECN-enabled TCP connections.

    • net.inet.tcp.cc.newreno.beta and net.inet.tcp.cc.newreno.betaecn set the multiplicative window decrease factor, specified as a percentage, applied to the congestion window in response to a loss-based or ECN-based congestion signal respectively. They default to the values specified in the draft i.e. beta=50 and betaecn=80.

    • net.inet.tcp.cc.abe_frlossreduce defaults to 0 (disabled) and can be set to non-zero to enable the use of standard beta (50% by default) when repairing loss during an ECN-signalled congestion recovery episode. It enables a more conservative congestion response and is provided for the purposes of experimentation as a result of some discussion at IETF 100 in Singapore.

    The values of beta and betaecn can also be set per-connection by way of the TCPCCALGOOPT TCP-level socket option and the new CCNEWRENOBETA or CCNEWRENOBETA_ECN CC algo sub-options.

    Submitted by: Tom Jones tj@enoti.me Tested by: Tom Jones tj@enoti.me, Grenville Armitage garmitage@swin.edu.au Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D11616 ```

    Meltdown-mitigation syspatch/errata now available

    The recent changes in -current mitigating the Meltdown vulnerability have been backported to the 6.1 and 6.2 (amd64) releases, and the syspatch update (for 6.2) is now available.

    ``` Changes by: bluhm@cvs.openbsd.org 2018/02/26 05:36:18 Log message: Implement a workaround against the Meltdown flaw in Intel CPUs. The following changes have been backported from OpenBSD -current.

    Changes by: guenther@cvs.openbsd.org 2018/01/06 15:03:13 Log message: Handle %gs like %[def]s and reset set it in cpu_switchto() instead of on every return to userspace.

    Changes by: mlarkin@cvs.openbsd.org 2018/01/06 18:08:20 Log message: Add identcpu.c and specialreg.h definitions for the new Intel/AMD MSRs that should help mitigate spectre. This is just the detection piece, these features are not yet used. Part of a larger ongoing effort to mitigate meltdown/spectre. i386 will come later; it needs some machdep.c cleanup first.

    Changes by: mlarkin@cvs.openbsd.org 2018/01/07 12:56:19 Log message: remove all PG_G global page mappings from the kernel when running on Intel CPUs. Part of an ongoing set of commits to mitigate the Intel "meltdown" CVE. This diff does not confer any immunity to that vulnerability - subsequent commits are still needed and are being worked on presently. ok guenther, deraadt

    Changes by: mlarkin@cvs.openbsd.org 2018/01/12 01:21:30 Log message: IBRS -> IBRS,IBPB in identifycpu lines

    Changes by: guenther@cvs.openbsd.org 2018/02/21 12:24:15 Log message: Meltdown: implement user/kernel page table separation. On Intel CPUs which speculate past user/supervisor page permission checks, use a separate page table for userspace with only the minimum of kernel code and data required for the transitions to/from the kernel (still marked as supervisor-only, of course): - the IDT (RO) - three pages of kernel text in the .kutext section for interrupt, trap, and syscall trampoline code (RX) - one page of kernel data in the .kudata section for TLB flush IPIs (RW) - the lapic page (RW, uncachable) - per CPU: one page for the TSS+GDT (RO) and one page for trampoline stacks (RW) When a syscall, trap, or interrupt takes a CPU from userspace to kernel the trampoline code switches page tables, switches stacks to the thread's real kernel stack, then copies over the necessary bits from the trampoline stack. On return to userspace the opposite occurs: recreate the iretq frame on the trampoline stack, switch stack, switch page tables, and return to userspace. mlarkin@ implemented the pmap bits and did 90% of the debugging, diagnosing issues on MP in particular, and drove the final push to completion. Many rounds of testing by naddy@, sthen@, and others Thanks to Alex Wilson from Joyent for early discussions about trampolines and their data requirements. Per-CPU page layout mostly inspired by DragonFlyBSD. ok mlarkin@ deraadt@

    Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:18:59 Log message: The GNU assembler does not understand 1ULL, so replace the constant with 1. Then it compiles with gcc, sign and size do not matter here.

    Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:27:14 Log message: The compile time assertion for cpu info did not work with gcc. Rephrase the condition in a way that both gcc and clang accept it.

    Changes by: guenther@cvs.openbsd.org 2018/02/22 13:36:40 Log message: Set the PG_G (global) bit on the special page table entries that are shared between the u-k and u+k tables, because they're actually in all tables.

    OpenBSD 6.1 errata 037 ```

    • 6.2

    ``` Changes by: bluhm@cvs.openbsd.org 2018/02/26 05:29:48 Log message: Implement a workaround against the Meltdown flaw in Intel CPUs. The following changes have been backported from OpenBSD -current.

    Changes by: guenther@cvs.openbsd.org 2018/01/06 15:03:13 Log message: Handle %gs like %[def]s and reset set it in cpu_switchto() instead of on every return to userspace.

    Changes by: mlarkin@cvs.openbsd.org 2018/01/06 18:08:20 Log message: Add identcpu.c and specialreg.h definitions for the new Intel/AMD MSRs that should help mitigate spectre. This is just the detection piece, these features are not yet used. Part of a larger ongoing effort to mitigate meltdown/spectre. i386 will come later; it needs some machdep.c cleanup first.

    Changes by: mlarkin@cvs.openbsd.org 2018/01/07 12:56:19 Log message: remove all PG_G global page mappings from the kernel when running on Intel CPUs. Part of an ongoing set of commits to mitigate the Intel "meltdown" CVE. This diff does not confer any immunity to that vulnerability - subsequent commits are still needed and are being worked on presently.

    Changes by: mlarkin@cvs.openbsd.org 2018/01/12 01:21:30 Log message: IBRS -> IBRS,IBPB in identifycpu lines

    Changes by: guenther@cvs.openbsd.org 2018/02/21 12:24:15 Log message: Meltdown: implement user/kernel page table separation. On Intel CPUs which speculate past user/supervisor page permission checks, use a separate page table for userspace with only the minimum of kernel code and data required for the transitions to/from the kernel (still marked as supervisor-only, of course): - the IDT (RO) - three pages of kernel text in the .kutext section for interrupt, trap, and syscall trampoline code (RX) - one page of kernel data in the .kudata section for TLB flush IPIs (RW) - the lapic page (RW, uncachable) - per CPU: one page for the TSS+GDT (RO) and one page for trampoline stacks (RW) When a syscall, trap, or interrupt takes a CPU from userspace to kernel the trampoline code switches page tables, switches stacks to the thread's real kernel stack, then copies over the necessary bits from the trampoline stack. On return to userspace the opposite occurs: recreate the iretq frame on the trampoline stack, switch stack, switch page tables, and return to userspace. mlarkin@ implemented the pmap bits and did 90% of the debugging, diagnosing issues on MP in particular, and drove the final push to completion. Many rounds of testing by naddy@, sthen@, and others Thanks to Alex Wilson from Joyent for early discussions about trampolines and their data requirements. Per-CPU page layout mostly inspired by DragonFlyBSD.

    Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:18:59 Log message: The GNU assembler does not understand 1ULL, so replace the constant with 1. Then it compiles with gcc, sign and size do not matter here.

    Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:27:14 Log message: The compile time assertion for cpu info did not work with gcc. Rephrase the condition in a way that both gcc and clang accept it.

    Changes by: guenther@cvs.openbsd.org 2018/02/22 13:36:40 Log message: Set the PG_G (global) bit on the special page table entries that are shared between the u-k and u+k tables, because they're actually in all tables.

    OpenBSD 6.2 errata 009 ```

    iXsystems

    a2k18 Hackathon Report: Ken Westerback on dhclient and more

    Ken Westerback (krw@) has sent in the first report from the (recently concluded) a2k18 hackathon:

    ```

    Whew.

    Once in Dunedin the hacking commenced. The background was a regular tick of new meltdown diffs to test in addition to whatever work one was actually engaged in. I was lucky (?) in that none of the problems with the various versions cropped up on my laptop. ```

    ``` I worked with rpe@ and tb@ to make the install script create the 'correct' FQDN when dhclient was involved. I worked with tb@ on some code cleanup in various bits of the base. dhclient(8) got some nice cleanup, further pruning/improving log messages in particular. In addition the oddball -q option was flipped into the more normal -v. I.e. be quiet by default and verbose on request.

    More substantially the use of recorded leases was made less intrusive by avoiding continual reconfiguration of the interface with the same information. The 'request', 'require' and 'ignore' dhclient.conf(5) statement were changed so they are cumulative, making it easier to build longer lists of affected options.

    I tweaked softraid(4) to remove a handrolled version of duid_format().

    I sprinkled a couple of M_WAITOK into amd64 and i386 mpbios to document that there is really no need to check for NULL being returned from some malloc() calls.

    I continued to help test the new filesystem quiescing logic that deraadt@ committed during the hackathon.

    I only locked myself out of my room once!

    Fueled by the excellent coffee from local institutions The Good Earth Cafe and The Good Oil Cafe, and the excellent hacking facilities and accommodations at the University of Otago it was another enjoyable and productive hackathon south of the equator. And I even saw penguins.

    Thanks to Jim Cheetham and the support from the project and the OpenBSD Foundation that made it all possible ```

    Poetic License

    I found this when going through old documents. It looks like I wrote it and never posted it. Perhaps I didn’t consider it finished at the time. But looking at it now, I think it’s good enough to share. It’s a redrafting of the BSD licence, in poetic form. Maybe I had plans to do other licences one day; I can’t remember.

    I’ve interleaved it with the original license text so you can see how true, or otherwise, I’ve been to it. Enjoy :-)

    ``` Copyright (c) , All rights reserved.

    Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: ```

    You may redistribute and use – as source or binary, as you choose, and with some changes or without – this software; let there be no doubt. But you must meet conditions three, if in compliance you wish to be.

    1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

    The first is obvious, of course – To keep this text within the source. The second is for binaries Place in the docs a copy, please. A moral lesson from this ode – Don’t strip the copyright on code.

    The third applies when you promote: You must not take, from us who wrote, our names and make it seem as true we like or love your version too. (Unless, of course, you contact us And get our written assensus.)

    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

    One final point to be laid out (You must forgive my need to shout): THERE IS NO WARRANTY FOR THIS WHATEVER THING MAY GO AMISS. EXPRESS, IMPLIED, IT’S ALL THE SAME – RESPONSIBILITY DISCLAIMED.

    WE ARE NOT LIABLE FOR LOSS NO MATTER HOW INCURRED THE COST THE TYPE OR STYLE OF DAMAGE DONE WHATE’ER THE LEGAL THEORY SPUN. THIS STILL REMAINS AS TRUE IF YOU INFORM US WHAT YOU PLAN TO DO.

    When all is told, we sum up thus – Do what you like, just don’t sue us.

    Beastie Bits

    Tarsnap ad

    Feedback/Questions
    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
    Thu, 29 Mar 2018 08:00:00 -0700
    -
    -
    (基於 PinQueue 指標)
    0 則留言