Initial Prototyping

The primary focus for the first stage of the project was to start prototyping a basic p2p communication system to get both peer discovery working and some basic communication between peers established. These goals have been reached and work is now starting on stage #2.

As of now, progress has been made towards a rudimentary prototype library in Rust that will serve as a reference implementation for the protocol and function as a library that other projects can import to use the protocol.

Currently, peer discovery using the mDNS protocol has been implemented, along with a basic request-response message system. Further details are provided below.

Prototype Implementation #

Most of the work so far has been on establishing a prototype of the initial reference implementation for the protocol, to get an idea of how peer-to-peer communication works and to gain more experience with networking in general. This way, the protocol can be developed in tandem with the reference implementation and adjustments can be made in real-time as specific requirements or limitations are identified.

As of this update, the prototype consists of a wrapper library around the libp2p Rust crate, as it provides flexible abstractions over the networking stack and an easy interface for developing a peer-to-peer system without needing to start from scratch. This takes the form of a Server struct, which controls the libp2p Swarm (the object that represents the p2p-network the peer is part of) and tracks the state of the network. Currently, the Server is capable of discovering peers using libp2p‘s implementation of mDNS, querying these peers to identify any hosted playlists, requesting the addition of songs to the playlists, and responding appropriately to these requests. The playlists and songs are currently dummy values that have no actual functionality, as at this stage of the project the focus has been on getting the network behaviour established. To allow the use of this library in other projects, it also makes use of a Client interface, which the Server makes use of as an event handler for relevant network events. On creation, a Server instance must be provided an object which implements Client, and it will call the methods defined in Client for network events that require application input. For example, if an AddSong request is received by the Server, it will call the Client‘s add_song method to fulfill this request, and respond with a success or failure according to the return value.

From this, an extremely simple test client was created which reads commands from stdin and sends the corresponding message to a connected peer, providing a basis for testing and experimenting with protocol functionality.

The next step for the prototype implementation will be to expand this client to actually implement functional playlists, so testing can be done on the coordination of playlist state between peers.

Protocol #

The first steps have been taken towards the protocol, with prototype messages designed for initial communication and coordination between peers, as this is t/he easiest to test. Currently, the protocol uses JSON-encoded messages with a request-response type behavior, as the majority of communication between peers in this type of network is intended to be many-to-one with a small number of playlist hosts (often only one), and primarily just simple message passing. The idea is that the requirement that every request expects a confirmation response will help reduce the change of playlist state becoming desynchronized. If a peer requests for another to add a song to it’s playlist, and it responds indicating success, that peer can then fully expect that song to have been added. Since this protocol is meant to only operate over LAN, it won’t have to make as many accommodations for packet loss or network delay.

An initial draft of the protocol has been made, of which only the ones that are currently present in the reference implementation are detailed here. A full specification will be given in the final report. The draft includes messages for requesting playlist state, adding songs, and requesting song files from peers. As these are implemented in the reference library, many adjustments and revisions are expected to be made.

The three initial request messages and their corresponding responses that are necessary for the system to function, and the ones currently implemented in their initial form in the reference library are:

1. RequestPlaylists, which is sent to request a list of any playlists hosted by a peer. The primary use is to contact newly discovered peers to identify if they have any new playlists. The expected response is the corresponding HostedPlaylists which is sent by the contacted peer in response. The contents of a HostedPlaylists message is a list of playlist IDs.
2. RecordPlaylist, which requests that a peer records that the sender has started hosting a new playlist with the identifier contained in the message body. The corresponding response is PlaylistAdded, with the body of this message being a boolean success flag and an optional message, which can be used to report errors, among other things.
3. AddSong, which requests that a peer add a new song to a playlist they are hosting. The content of this message currently consists of two fields: PlaylistId and SongId. On receipt of this request, peers respond with SongAdded. If the song was successfully added, this message will contain success: true, otherwise this will be false and the optional message field providing an explanation.

In their current form, these messages are only for testing and prototyping, and will be expanded and elaborated upon as necessary functionality is identified going forward. These will most likely not exist unchanged over the course of the project, especially the response messages, of which more detailed and specific messages will be designed. After initial testing and deliberation, the use of specific response types for each request is probably not necessary, and instead a limited set of success and failure messages will be used instead. For example, when a peer receives an AddSong request, they should instead respond with either Success or a specific error messages explaining the reason, such as UnkownPlaylist. Initial revisions to the protocol have been made so that only requests that are expected to return specific content, such as RequestPlaylists, will require unique responses, and requests which only require confirmation of an action being taken will use one generic Success message. Specific Failure messages will be designed as needed.

Next Steps #

For the next stage of the project, the focus will be on filling out and expanding the reference library to include an example application that will be able to create simple playlists from a fixed set of songs. This will first require expanding the reference library to support the remaining portions of the protocol. These goals will entail:

• Expanding the Server implementation with data structures to track and manage playlist state and coordinate this state with peers.
• Expand the Client interface with additional methods needed by Server to provide this functionality.
• Create a Playlist and Song interface which applications can implement on their corresponding data structures so they can be understood by Server.
• Identify shortcomings or missing components of the protocol that are needed to complete this implementation.
• Revise and update the protocol as needed.