GopherCon 2018 - Implementing a Network Protocol in Go
Presenter: Matt Layher
Liveblogger: Beyang Liu
A very detailed walkthrough of implementing a networking protocol (NDP in IPv6) in Go, with many, many code snippets.
Matt Layher (@mdlayher, talks) is an engineer at Digital Ocean.
Intro
- The IPv6 Neighbor Discovery Protocol (NDP) is our focus for today
- IPv6 is an important step for the Internet
- Go can be used for many low-level networking applications
Outline:
- Introduction to IPv6 and NDP
- Using NDP with Go: github.com/mdlayher/ndp
- Building and testing network protocol packages
Intro to IPv6
IPv6 adoption:
What is IPv6?
- The next generation of the Internet Protocol
- Draft standard in RFC 2460 (December 1998!)
- 128 bit IP addresses, huge improvement over 32 bit IPv4 addresses:
- IPv6: 2^128 addresses, but not all used for hosts
- IPv4: 2^32 addresses
How is IPv6 different from IPv4?
- Wire format simplified, more extensible
- Residential ISPs can offer entire IPv6 prefixes instead of 1 IPv4 address:
- IPv6: 2001:db8:abcd:ffff::/64: 2^64 addresses
- IPv4: 192.0.2.10/32: 1 address
IPv6 tips and tricks
- Many shell utilities have a “-6” flag to use IPv6
- A useful website for testing IPv6 configuration: ipv6-test.com
- My favorite ping target:
Intro to NDP
What is NDP?
- Effectively the IPv6 equivalent to IPv4’s ARP
- Runs on top of IPv6 + ICMPv6 with link-local addresses: fe80::/10
- Used to ask a network neighbor for its MAC address using IPv6 address
- A: Who has “B”? Tell “A”.
- B: “B” is at “04:18:d6:a1:ce:b7”.
IPv6 and NDP’s big advantage:
- DHCP is not usually necessary to configure globally-routable IPv6 addresses:
- Stateless Address Autoconfiguration (SLAAC) via NDP
- No DHCPv6 required whatsoever
- SLAAC + Stateless DHCPv6
- Addresses via SLAAC, more configuration via DHCPv6
- Stateful DHCPv6
- All addresses and information from DHCPv6
- Stateless Address Autoconfiguration (SLAAC) via NDP
SLAAC via NDP
- SLAAC uses NDP router advertisements to provide IPv6 prefix information
- “A” sends a router solicitation
- “R” sends a router advertisement:
- Prefix “P::/64”, use SLAAC, valid for 24 hours
- “P:76d4:35ff:fee7:cbc4” computed and assigned
NDP and Go
- Your operating system usually handles NDP; why is it useful for Go programs?
- github.com/mdlayher/ndp: Go package for using NDP
Package ndp overview
- Primary types:
ndp.Message
interface: marshaling/unmarshaling of NDP messagesndp.Option
interface: marshaling/unmarshaling of NDP optionsndp.Conn
struct: manage ICMPv6 connection, read/writendp.Message
s
How do we go from bytes to a complete NDP package?
From bytes to messages
NDP message basics:
- ICMPv6 header determines which NDP message is used
- Type specifies NDP message, Code always 0
- Initial NDP messages and options defined in RFC 4861
- Fixed length messages, variable options
Parsing bytes:
- An ICMPv6 header will always precede an NDP message
- NDP messages on their own are not useful without the ICMPv6 header
- Exporting marshal/unmarshal methods bloats the API and GoDoc
- Solution: add functions which always add/remove the ICMPv6 header
ndp.Message
interface
- Exported Type method for documentation, but other methods unexported
ndp.ParseMessage
ndp.ParseMessage
function does bounds checking validation, determines concrete type, continues parsing:
Bounds checking: when using slice elements, you must perform bounds checks to avoid panics:
Determining ndp.Message
type: use a switch to choose the right interface implementation:
Unmarshal the ndp.Message
implementation: call into the type’s methods to do the rest of the work, skipping the header
A couple of comments on the parsing logic:
- Using
ndp.ParseMessage
, it’s easy to parsendp.Message
types - Concise API: one parsing function
- Correctness and simplicity first, performance optimizations later
ndp.Message
implementation
Our first - Neighbor Solicitation (NS) messages ask a machine for its MAC address
- For now,
ndp.Option
is unimplemented
What an ICMPv6 + NDP NS message looks like:
The ndp.NeighborSolicitation
type mimics the structure defined by the RFC, using doc comments to provide references:
(Neat godoc feature: it will automatically hyperlink to the RFC as defined above in the comments.)
Checking for IPv4 and IPv6 addresses:
- net.IP can contain IPv4, IPv6, or totally invalid IP addresses
- A combination of To4 and To16 methods determine the actual type
- Don't think this is the friendlies API. Something I’d love to see improved upon in Go 2
- net.IP interface? net.IPv4 and net.IPv6 types?
checkIPv6
function:
ndp.NeighborSolicitation
unmarshaling validates incoming bytes and replaces the structure all at once:
To validate byte inputs, ensure that field values make sense, typically using rules defined by RFC. I.e., verify that we don't have any sneaky IPv4 addresses:
To replace the structure while unmarshaling, (1) dereference the pointer and replace contents with completed structure. (2) Always make a copy of data from the input slice; don’t assume it’s safe to retain:
From messages to bytes
Things to remember when marshaling messages:
- An ICMPv6 header will always precede an NDP message
- NDP messages on their own are not useful without the ICMPv6 header
- Do the parsing operation in reverse
ndp.MarshalMessage
function
Marshal an ndp.Message into binary, prepend ICMPv6 header
When marshaling ndp.Messages
, simplicity wins. Allocating is okay until your performance needs are not met:
Same goes for ICMPv6 messages:
ndp.NeighborSolicitation
marshaling
Validate before you allocate:
Don’t bother allocating memory until you've checked your inputs:
Allocate once, if possible (allocating once is ideal for speed, but keep it simple):
When allocating memory...
- Simplicity wins - allocating is okay!
- Write comprehensive unit tests to lock in your behavior
- Measure for bottlenecks using Go benchmarks and pprof
- Optimize only after finding evidence of performance issues
ndp.Message
API
ndp.Message
types:
ndp.Message
usage:
From bytes to options
NDP option basics
Options are encoded in type, length, value (TLV) format:
- Fixed length: type
- Fixed length: length
- Variable length: value/data
TLV options:
ndp.Option interface
Parsing options:
- An NDP message will always precede options
- NDP options on their own are not useful without an NDP message
- Exporting marshal/unmarshal methods bloats the API and GoDoc
- Solution: use unexported functions with
ndp.ParseMessage
andndp.MarshalMessage
- Solution: use unexported functions with
marshalOptions function
parseOptions function
Parsing is just a little trickier:
ndp.Option
types include the following:
- Source/target link-layer address
- MTU
- Prefix information
- Recursive DNS server
- … and more! If we implemented all of them, our API could bloat quickly!
Tips for implementing options:
- Consider only implementing the most common options in your package
- Prevent API bloat, support 90% of use cases
- Tip: add a “raw option” type or similar to enable further extension
ndp.RawOption type
It directly exposes the TLV fields, so code outside this package can pass options that aren't defined in this package. If a particular type of option is used often enough, we can add first-class support for it later.
ndp.Option
types:
ndp.Option
usage:
Fuzzing byte parsers
Fuzzing lets you catch and prevent errors arising from unexpected and unhandled input cases. E.g., avoid errors like this one:
Enter Dmitry Vyukov's go-fuzz
. If you’re parsing raw bytes, there's a high potential for unexpected behavior:
- Bad input causing application problems
- Possibility of a panic taking down your program!
github.com/dvyukov/go-fuzz address this problem:
- Throws arbitrary bytes at your parser and finds crashers!
- Mark inputs as “interesting” or not to guide fuzzer
go-fuzz setup:
go-fuzz usage:
- Prepare the fuzzer by building an instrumented test program:
- Run go-fuzz with multiple CPU's and output results to ./fuzz/
- Inspect the resulting crasher inputs
- Write a test, fix the bug, and repeat!
go-fuzz conclusions: Use it! Use go-fuzz on ALL byte parsers: github.com/dvyukov/go-fuzz
- Find parsing problems now, not during a 3am outage
- A multi-worker mode is available for use with clusters of machines
- Submit “trophies” to the go-fuzz README
ndp.Conn
Let's implement the struct that represents an NDP connection.
“Conn” types represent network connections. They typically have the following:
- “Dial” and/or “Listen” constructors
- “Close” to free resources
- “Read” and “Write” to pass messages
net
vs x/net
- NDP is transported over IPv6 + ICMPv6
- Standard library net doesn't quite provide all the functionality we need
- golang.org/x/net is designed for advanced use-cases!
ICMPv6 networking packages in Go:
- golang.org/x/net/icmp
- golang.org/x/net/ipv6
- Huge shout-out to Mikio Hara for his work on these packages and countless other low-level networking packages for Go
Here's how you create a ICMPv6 listener (this is a privileged operation, usually requires root):
Reading ICMPv6 messages is similar to standard APIs, but also returns IPv6 control messages:
Writing ICMPv6 messages is similar to standard APIs, but you can specify IPv6 control messages:
ndp.Conn usage:
Create an ndp.Conn by selecting an interface, dialing ICMPv6, and specifying an address to listen on:
How to read messages? Here's a code snippet to keep reading and printing messages until an error occurs:
Writing ndp.Messages: Send a router solicitation to trigger router advertisements on the network:
Build a tool to test your package
Add a cmd/ directory with a testing utility during development
- Consider building it out to become a useful tool
- cmd/ndp: tool for generating and capturing NDP traffic
Introducing the ndp tool:
Easier to use than tcpdump
. E.g., here's the tcpdump
command and output to watch for NDP packets:
Compare that with ndp
:
Troubleshooting your ISP’s equipment with Go
You can also use Go to troubleshoot any difficulties your ISP has with IPv6.
Ubiquiti EdgeRouter Lite can run Go programs:
- No luck with tech support: “your WiFi router isn’t working”
- … a modem swap during an upgrade made the problem disappear
Conclusions
- IPv6 is great, check out ipv6-test.com to see if you’re using it
- Network protocols are powerful building blocks
- Go is an excellent language for exploring low-level network protocols
- Build tools to solve real problems on your network!
Resources:
About the author
Beyang Liu is the CTO and co-founder of Sourcegraph. Beyang studied Computer Science at Stanford, where he published research in probabilistic graphical models and computer vision at the Stanford AI Lab. You can chat with Beyang on Twitter @beyang or our community Discord.