Protocol Revisions

Development of the reference implementation for the protocol ended up stalling somewhat due to the complexity of the initial protocol draft. To address this, major revisions to the protocol were made for a second draft in order to cut down the functionality into a more manageable scope for the project. Many of the capabilities described by the initial draft were more complex than could feasibly be implemented during the course, this expanded functionality could instead be provided by later extensions to the protocol.

The Revised Protocol #

The initial drafts of the protocol allowed for peers to be able to host multiple playlists, communicate playlist state between all peers, and for peers to request modifications to playlists beyond simply adding songs (removal, re-ordering, etc.) While simple to design at the protocol level, this was more complex than expected to implement, as it would require systems in place to ensure all peers correctly shared and communicated playlist state without de-sync and some form of unique ID allocation for playlists and songs to ensure peers could properly communicate playlist modification requests to each other. Furthermore, this was also beyond the scope of deliverables from the project proposal, which only detailed the capacity for peers to request songs be added to playlist.

As such, the protocol has been cut back to only provide the core capabilities set out by the proposal, defined by three message types below. Note these are formatted using Rust syntax as they appear in the reference implementation and do not correspond to the actual message formats.

1. Request messages are sent between individual peers to request certain actions or information, these include:
   1. Request::Playlist, which requests the metadata of the playlist hosted by the receiver, if it is hosting a playlist.
   2. Request::AddSong(SongMetadata) is used by peers to request that the receiver add the song described by the SongMetadata object to their playlist. SongMetadata is a json dict containing enough information to uniquely identify the song. (Song title, artist, album, release date, etc.) The receiver is responsible for using this data to identify whether it has access to this song in it’s own library or not.
   3. Request::SongFile(SongMetadata) is used to request the actual audio file of a song from a peer. This is used by hosting peers, who would otherwise be unable to fulfill an AddSong request because they could not find the song in their own library. If this was the case, they send a SongFile request back to the peer which sent the AddSong.
2. Response messages are sent by peers in direct response to a request message, and all requests must have an according response.
   1. Response::Playlist(PlaylistMetadata) is sent in direct response to Request::Playlist containing a PlaylistMetadta json object containing the name and description of the playlist, and playlist state consisting of the number of songs and SongMetadata of the currently playing and next-up songs if they exist.
   2. Response::NoPlaylist is sent in response to Request::Playlist if the peer which received the request was not hosting a playlist when the request was recieved.
   3. Response::SongRejected is sent in response to a Request::AddSong message if the receiver either a) was not able to find the song in their own library or b) did not receive the song after sending a Request::SongFile message to the originating peer.
   4. Response::SongAccepted is sent in response to Request::AddSong if the song was added to the playlist successfully.
   5. Response::SongFile(song) is sent in response to Request::SongFile containing the file requested.
   6. Response::UnableToProvide is sent in response to Request::SongFile if the receiver cannot provide the song that was requested.
3. Broadcast messages are sent to all or some peers to inform them of relevant information.
   1. HostingPlaylist(PlaylistMetadata) announces that the sender is now hosing a playlist described by PlaylistMetadata and can accept songs.
   2. StoppedHosting announces that the sender has stopped hosting their playlist.
   3. PlaylistUpdate(PlaylistMetadata) announces that the state of the sender’s playlist has updated to the new state described in the included PlaylistMetadta object.

This revised and trimmed down protocol specification is much more in-line with the initial project proposal and significantly more feasible to implement.

Implementation Progress and Changes #

After running into limitations with the basic request-response system provided with libp2p and following some additional research, it was decided that Broadcast messages in the reference library will be built on top of the Gossipsub implementation provided by libp2p. Primarily a protocol for p2p chat applications, Gossipsub allows peers to subscribe to and post messages on topics. Messages posted on a topic are then broadcast to all subscribed peers. This greatly simplified the reference implementation, as now a playlist topic, subscribed to by all peers, can be used to announce when peers are hosting/stopping hosting their playlists, and peers now create Gossipsub topics for their playlists which they use to broadcast state changes on. The only limitation of this approach is that it greatly hinders the flexibility of the reference library, as it would not be able to inter-operate with other implementations that do not use Gossipsub for message broadcasting. The actual protocol however, does not require or specify a specific method of sending broadcast messages, and the reference library would be updated and revised with a more universal broadcasting capability for real-world use if this project were to continue beyond the scope of this course project, but Gossipsub works well for demonstration purposes and simplifies development.

Otherwise, the Server described in the first update has been almost entirely complete, with the most difficult part—the network event loop and message handling—fully functional. This means that the work going forward will be easier and more straightforward as it just involves adding additional match cases for the remaining message types.

The Client implementation has also been revised alongside the protocol changes to reduce the complexity that would be required when implementing a Client. The current iteration of the Client trait is as follows:

use crate::types::*;

pub trait Client {
    // Methods for the Server to get information about the Client.

    fn hosted_playlist(&self) -> Option<PlaylistMetadata>;

    // Methods for the Server to notify the Client of changes to playlists.

    /// A new playlist has been discovered in the network.
    fn playlist_discovered(&self, playlist: PlaylistId);

    /// A playlist has been removed from the network.
    fn playlist_removed(&self, playlist_id: PlaylistId);
    /// The subscribed playlist's state has changed.
    fn subscribed_playlist_update(&self, playlist_data: PlaylistMetadata);
    /// A peer has requested to add a song to the playlist
    fn add_song_request(&self, song: SongMetadata) -> Result<(), Error>;
}

The only additional function which will need to be added is song_request(&self, song: SongMetadata) -> Option<SongData> to fulfill Request::SongFile, and then the only thing that would be required to implement in order to use this library is an implementation of Client according to this specification, which would be provided to an instance of Server.

Next Steps #

The next stage of the project will consist of finalizing the reference implementation and implementing the remaining functionality of the protocol, as handling Request::SongFile and related messages is not yet fully completed due to the time spent on revising the protocol.

The last major deliverable to work on is to expand the current SampleClient to include a small library of songs that it can use and actual playlists beyond the hard-coded test systems. Essentially this will be implementing an extremely basic interface to demonstrate practical use of the protocol beyond the hard-coded test interactions that have been implemented.

Work on a test suite was also started, which will be expanded upon as well.