English Version

MIDIbox Hardware Platform, USB PIC Module

This module was planned as a cheap and easy alternative solution to the MBHP_USB module. It is based on the PIC18F4550 (or PIC18F2550) which is available for less than 6 US$.
Meanwhile a better solution has been found: MBHP_USB_GM5

This module is expired as USB is natively supported by the MBHP_CORE_LPC17 module!

Summary

History - MBHP_USB_PIC firmware, first attempt

Initially the implementation of the firmware was planned as a nice one-day project, as I didn't expect much effort for doing a port of the Cypress based MBHP_USB to the PIC architecture, which I know pretty well. The USB descriptors for USB MIDI, the MIDI parser, the interrupt handlers for MIDI Rx/Tx were ready to be re-used.

So, at a rainy weekend in March 2005 I downloaded the USB framework from the Microchip homepage, installed the USB bootloader (for quick program updates), and integrated the basic MIDI handling code into the CDC example. During the first tests I already noticed that the example code didn't work so stable as expected. Under stress conditions, received MIDI data got lost, and transmitted MIDI data was sporatically corrupted.

After some hours I couldn't believe anymore that this was caused by my own code, so I started to debug the transmitting issue first by writing a short program, which just forwards incoming MIDI data to the output port. Even this minimal code was failing: on large SysEx dumps, sometimes more data was forwarded than received. We know that data can get lost when a "serial forwarder" doesn't send out the incoming stream with at least the same speed (bottleneck issue), but how can bytes be added into the stream when the software is primitive enough, so that programming errors can be excluded?

Intuitively I tested the same code on a "known good" device - the PIC18F452. It was working fine! I assumed a physical issue (Device overclocked? Voltage unstable? Untested silicon with defects? Noisy Rx/Tx line?), however - three PIC18F4550 and three PIC18F4520 were showing the same effecs on different prototype boards. I read about the new EUSART peripheral and assumed software incompatibilities, but according to datasheets it should behave identical to the old version. I searched in the errata sheets for possible errors, but even the documented interrupt bug wasn't relevant, as I wasn't using interrupts anymore at this time. So I continued to analyse the effects by creating a test scenario with two chips: a PIC18F452 which sends and receives data to/from a PIC18F4550, and which toggles a pin on mismatches so that a wrongly transmitted byte can be captured with a scope. Later I also added an "dynamic delay" into the forwarding routine running on PIC18F4550 to emulate jitter which is caused by interrupt handlers in realworld applications. This setup allowed me to reproduce the issue very easily, and to visualize the additionally sent bytes on a scope.

To make it short - the additional bytes were sent due to a silicon bug in the new EUSART peripheral. See also this article in the Microchip forum, where I reported my observations. I had to contact the Microchip support multiple times to point out, that this is a silicon bug which exists in many derivatives which were produced in the past. Finally they decided to add an hint into the errata sheets of some (but not all) affected devices. It describes the defect and a possible software workaround.

Some words to the suggested workaround (see errata sheet, bug #19): both are not acceptable for my usecases due to a potential risk for buffer overflows at the receiver side while streaming data. Especially for MIOS such workarounds are not applicable, as the performance is affected too much, and as there are not free timer resouces.

So, what was the result: I stopped the firmware development after it was clear that there is a silicon bug, especially because I wasn't able to ensure anymore, that even if this bug should ever be fixed in later revisions, somebody is able to determine if a mailorder company like Mouser or Reichelt offers the "good" or "bad" one.

Regardless of this bad experience, I followed the progress at Microchip in the following months, and infrequently tested new chip revisions of PIC18F4550, PIC18F4520 and PIC18F4620 by ordering samples. Between 2005 and 2007 all devices were affected, even when the bug was not listed in the errata sheet of the appr. chip revision. Microchip support didn't really react on such inconsistencies (statement: "bug will be fixed in future"). They still haven't managed their documentation issues.

FYI: December 2007 I got a PIC18F4550 rev.5 where the EUSART bug is finally fixed.

History - MBHP_USB_PIC firmware, second attempt

One year later, in spring 2006, I continued with the firmware; as a feasible workaround I planned the usage of an external MIDI transmitter/receiver connected via IIC - the MBHP_IIC_MIDI module. I rewrote the complete firmware in assembly language, as the 3-month evaluation license of MPLAB-C18 compiler already timed out, and since I wanted to ensure that everybody is able to compile the firmware under Windows/Linux/MacOS without buying expensive tools. Also performance reasons lead to this decition - there was a high risk that code generated by the freely available SDCC compiler was too slow for handling 5 MIDI In and 4 MIDI Out ports.

Yes, you are reading correctly: the intention was to handle 4 MBHP_IIC_MIDI modules + the working "internal" EUSART based MIDI In port from a single USB device:

In general the firmware was running fine on my Windows and Linux PC, but I wasn't happy about the windows legacy driver implementation for the USB MIDI protocol, since it doesn't come with multiclient capabilities. I was mainly working under Windows this time, e.g. to develop and upload new firmware with MIOS Studio while having MIDI-Ox and Reaktor or Logic Audio running in parallel - this is a strong requirement for my workflow.

Another issue I noticed was unexpected data loss during transfers of large SysEx dumps, which happens when outgoing data is loopbacked from MIDI Out to the MIDI In port. I called this effect "blocked pipes" in this german article, as it describes the effect pretty well: under heavy load the IN pipe is sporatically blocked for several miliseconds if the OUT pipe is not serviced immediatly after the host has sent new data. The following scope snapshot illustriates this effect:

The lower channel toggles whenever the IN/OUT endpoint handler (USBCLS_EP2_Handler) is called, and the upper channel mirrors the BD2_STAT.UOWN flag, which is set so long the SIE owns the IN buffer.

This effect can cause buffer underruns at the MIDI receiver site. I doublechecked this by replacing each third byte of a SysEx stream by the number of free bytes in the receive buffer:

This "trace" shows, that a) much less bytes are received than transmitted, and b) that there are dramatic jumps in the number of free bytes. E.g. 0x7f -> 0x5a means: we had to buffer 37 bytes. At a baudrate of 32150, this means, that the IN endpoint was blocked for ca. 11 mS - and this time is matching with the waveforms on the scope!
Since this happens multiple times during the transfer of a SysEx stream, the receive buffer will underrun very quickly and cause data loss. Note: when MIDI In and Out are directly loopbacked!

It doesn't happen when MIDI In/Out are connected to a MIDI device which receives/transmits SysEx data quasi unidirectional. E.g., when large SysEx dumps are sent from one MIDI interface to MBHP_USB_PIC, or vice versa, no data will get lost.

Btw.: this effect even happens, when the firmware skips USBCLKS_EP2_Hander_Out so long the receiver buffer has to be emptied. So, preventing that packages will be forwarded to MIDI Out so long something is received doesn't help.

It still isn't clear to me, if this is a PIC or Windows related issue. I wasn't able to reproduce it under Linux at this time, but the reason could be, that the appr. USB drivers are running under different timing conditions (e.g. slower or faster as the Windows driver...).

Interestingly I read in different forums, that USB devices from Korg, Behringer and ESI, which were using the legacy Windows driver as well, had similar issues with SysEx transfers. These companies provide alternative drivers, which are unfortunately protected and cannot be re-used for private projects just by adapting the USB descriptors. And another interesting point, which totally confuses me, is that I wasn't able to reproduce it with the AN2131SC based MBHP_USB interface.

Meanwhile I lost the interest on USB (again), and created the MIDI Router instead, which offers a much higher flexibility for my usecases, as this project allows to forward MIDI streams of different devices even without a computer and with best timings.

History - MBHP_USB_PIC firmware, third attempt: In the following months I frequently got EMails and forum requests concerning the MBHP_USB_PIC project. People intended to save money with this DIY solution, or they intended to use the firmware as a template for their own projects. I never found the motivation to release the project properly due to the reasons described above, and therefore distributed the firmware only as a learning example with a big warning about the known limitations.

Installation

For those who are not deterred from the bad news above, I describe the installation procedure here.

I strongly recomment you to install the Microchip USB bootloader, which can be found in the download section at the end of this page. The package contains a .hex file which has to be programmed into the PIC with a programmer like MBHP_BURNER. Once this has been done, you will be able to download updates of the MBHP_USB_PIC firmware with a comfortable Windows application very quickly.

Users of other operating systems have no benefit from this USB bootloader, and need to program the MBHP_USB_PIC firmware directly (e.g. under Linux with "pkp").

Once your core is up&running, Windows will ask you for a driver - it can be found in the "pc_usb_driver" directory of the bootloader package.

Now you can start the "Pdfsusb" application and load the firmware "project.hex", which can be found in the MBHP_USB_PIC release package:

Thereafter press the "Program Device", afterwards the "Execute" button of the application, or reset the PIC with the Reset Button of your board. Windows should re-connect to the USB device and find an "Audiodevice" (in german installations: "Audiogerät"). MIDI-Ox should show this device as well (if not: re-start MIDI-Ox if it was opened before!)

Now the PIC will always boot with the MBHP_USB_PIC firmware after a reset or power-on. Whenever you want to update the firmware, press the Bootloader Button together with the Reset Button, thereafter release (only) the Reset Button until Windows notifies you about the USB device exchange with a sound (this will happen immediately after reset). Now depress the Bootloader Button and select the "PICDEM FS USB 0 (Boot)" device again (selector will be empty so long you haven't done this) in the Pdfusb application, load the new .hex file and press the "Program Device" button.

If you are planning to connect MBHP_IIC_MIDI modules as IO extension, you need to recompile the firmware with different settings in the Makefile: set IIC_MIDI_MAX_NUMBER to 4 and NUMBER_MIDI_INTERFACES to 5. Thereafter type "make" (it's assumed that the MIOS8 toolchain is installed on your computer; SDCC and GPASM are required for compilation, it's basically the same toolchain which is used for C based MIOS applications.

A wiring diagram for the modules can be found here. Note that open RA0/1/2/3 input pins (which are not connected to the RI# line of a MBHP_IIC_MIDI module) need to be connected to Vdd (+5V), otherwise the firmware could behave randomly (slow down or crash), as it isn't able to determine anymore, if a module has received some data or not. This special requirement has only to be considered when IIC_MIDI_MAX_NUMBER has to be set to a value != 0 in MAKEFILE.SPEC - and this explains also, why I disabled this option by default.

Firmware Structure

This information is relevant for people who want to debug or enhance the firmware. The code is split over different modules for a clean seperation of functions. It's completely implemented in assembly language, so that everybody is able to compile the firmware under Windows/Linux/MacOS without buying expensive tools. SDCC is required to create the USB descriptor tables (it was the simplest way, as packed, non-padded arrays cannot be defined with GPASM). For extensions it could be possible to add C modules in future, but note that SDCC cannot generate performant code to handle data packages and buffers in such an efficient way like I did in the USB class driver.

Description of the most important files:

main.asm: start vectors, interrupt handler, initialisation routine and main loop.
Note that the start and interrupt vector is located at 0x000/0x008 (for usage without bootloader) and 0x800/0x808 (for usage with bootloader). It isn't required to switch between these two locations, as the bootloader will ignore the lower addresses.
usbdrv.asm: USB driver based on Microchip Framework, rewritten in assembly
usbdsc.c: USB descriptors for USB MIDI
usbcls.asm: USB MIDI Class Driver which can handle up to 5 IO ports
midi.asm: IO buffer handling for internal MIDI interface
int_midi.asm: MIDI parser and interface to usbcls.asm (USB MIDI Package Handlers)
iic.asm: IIC transfer routines
iic_midi.asm: Interface to usbcls.asm (USB MIDI Package Handlers)

Download

PCB data, can be viewed, modified and converted with Eagle Light. The .pdf based schematic has been created with xcircuit. There are no special eagle schematics available, since components have been netlisted in the .brd file directly!
Module	Schematic	Layout Data	Quick-view
MBHP_USB_PIC18F4550_V1	mbhp_usb_pic18f4550.pdf	-	-
MBHP_USB_PIC18F2550_V1	mbhp_usb_pic18f2550.pdf	-	-
MBHP_USB_PIC_WITH_IIC	mbhp_usb_pic_with_iic.pdf	-	-

Firmware and Bootloader
File	Size	Description
mbhp_usb_pic_v1_1.zip	42k	MBHP_USB_PIC firmware + source code
microchip_usb_bootloader.zip	988k	The USB bootloader which I downloaded from the Microchip Website anno 2005. I wasn't able to find it there anymore, but I guess that it's burried somewhere under... don't know... however, this version runs fine!

Is it possible to merge MIOS with the firmware?: No, this isn't possible due to resource conflicts (clock frequency, allocated IO pins, internal RAM areas dedicated for USB endpoint buffers). Just take MBHP_USB_PIC as a separate MIDI interface, and cross-connect the MIDI I/O ports to a MBHP_CORE (MBHP_USB_PIC MIDI In->MBHP_CORE MIDI Out, MBHP_USB_PIC MIDI Out->MBHP_CORE MIDI In). The usage of optocouplers is recommended to avoid ground loops between your PC and your audio gear!

Last update: 2023-11-04

MIDIbox Hardware Platform, USB PIC Module

Summary

History - MBHP_USB_PIC firmware, first attempt

History - MBHP_USB_PIC firmware, second attempt

History - MBHP_USB_PIC firmware, third attempt

Installation

Firmware Structure

Download

Is it possible to merge MIOS with the firmware?