Custom binary file format

Anyone know how I could save data as a custom binary file format in vvvv?

It’s to speed up file reading times on an arduino with sd card.

Initially I was using csv files from vvvv. Now I’m using hex, which shaves off 30%, but it’s be nice to go faster

I’m saving 512 8bit values in each file, as a ‘frame’ of data

what kind of 8 bit values?
if it’s just integers to max 255, you can convert it to byte via SpellValue (String) Mode Ascii and put that directly into Writer (File)

Hi Woei, thanks for getting back to me again.

That’s what I was thinking before but

‘because ascii char 0-31 are empty characters, 32 is space’

how does it work if there are empty characters?

edit - Are they there but just not visible?

double edit - Any idea how to convert an ascii character to a 0-255 integer in c++ / arduino? :)

hoi boni

ascii char 0-31 are just not visible by default. if you turn on Show Characters of a IOBox (String) in the inspector you’ll see though.

to read that back in it should be as simple as^code:
int var = Serial.read();
^
if you just send one, if not you have to split before casting to int.

eyyyy. all fixed.

Was as simple as int x = (int)char;

I’m reading ‘frames’ holding 512bytes each from an sd card to be sent over dmx. So it’s a kind of dmx player that can play back sequences recorded in vvvv.

Seems to be working fine, though only at 24fps currently. That’s for 512 values though (1 universe). If you need less it’ll be faster.

Let me know if the code is of interest to you and I’ll post it

hi boni,

just out of curiosity i’d be interested to have a look ;)

Hey motzi. Just noticed your reply.

It’s all working now, but not with 512 channels at a decent rate. Thank fully it’s main purpose initially was just to drive 1 channel!

I need to do a test to confirm, but I think you can get about 60 channels at 30-40 fps. Apparently the arduino (I’m using a mega) just aint fast enough for a full universe. I’m going to get one of the 32bit arduino clones and have a play. They should be able to handle it.

Anyway, here’s the code. You need sdfatlib, and dmxsimple libraries. It’s using a really fast method for file access which just reads the next file in the files system, so relies on the files being copied to the card in the right order (windows 7 seems to copy things alphabetically, so we’re ok)

Attached is a vvvv patch to write out frame files

The chip select pin must match the setup you’re using. let me know if you have problems.

/*
 * Open files in the root dir sequentially, parse, send 8bit dmx values
 */
- include <SdFat.h>
- include <DmxSimple.h>


// SD chip select pin
const uint8_t chipSelect = 10;

// file system object
SdFat sd;

SdFile file;

// define a serial output stream
ArduinoOutStream cout(Serial);
//------------------------------------------------------------------------------
void setup() {
  char name[13](13);
  uint8_t bufferOne[512](512);
  char in_char=0;
  int newByte=0;
  unsigned long time = 0;
  int fps = 30;

  DmxSimple.usePin(2);
  DmxSimple.maxChannel(3);

  Serial.begin(57600);

  // initialize the SD card at SPI_HALF_SPEED to avoid bus errors with
  // breadboards.  use SPI_FULL_SPEED for better performance.
  if (!sd.init(SPI_HALF_SPEED, chipSelect)) sd.initErrorHalt();


  while(1)
  {
      // open next file in root.  The volume working directory, vwd, is root
      while (file.openNext(sd.vwd(), O_READ)) 
      {
          time = millis();
          file.getFilename(name);
                   
          int byteCount = 1;
          while (1) 
          {
              
              newByte = file.read();
              
              if(newByte == -1)
              {
                break;
              }
             
              //Serial.println(newByte); //prints the bytes, if you want to see the values
              DmxSimple.write(byteCount, newByte);
              byteCount++;  
       
          }
          
          //cout << name << endl; //prints filename
          file.close();
          
          int tmp = [1000/fps)-(millis()-time](https://vvvv.org/documentation/1000/fps)-(millis()-time);
          delay(tmp);
          
      }
      //cout << "Done" << endl; //prints "done"
      
      sd.vwd()->rewind();
  }
}
//------------------------------------------------------------------------------
void loop() {}

edit - there are a couple of unused variables in there! see if you can find em ;)

frame gen.v4p (23.6 kB)