Glitches on texture output. short pause every 1or2 seconds

Hey all

I’m getting glitches with texture output in a dynamic texture node. I’m using a method which fills the whole texture simultaneously from an IntPtr.

Rather than post files that people would have to downlaod, it’s much easier just to paste this into a node (thanks VVVV group, CodeEditor!)

- region usings
using System;
using System.ComponentModel.Composition;
using System.Runtime.InteropServices;

using SlimDX;
using SlimDX.Direct3D9;
using VVVV.Core.Logging;
using VVVV.PluginInterfaces.V1;
using VVVV.PluginInterfaces.V2;
using VVVV.PluginInterfaces.V2.EX9;
using VVVV.Utils.VColor;
using VVVV.Utils.VMath;
using VVVV.Utils.SlimDX;

- endregion usings

//here you can change the vertex type
using VertexType = VVVV.Utils.SlimDX.TexturedVertex;

namespace VVVV.Nodes
{
	#region PluginInfo
	[PluginInfo(Name = "SpeedyTextureOut", Category = "EX9.Texture", Help = "Basic template which creates a texture", Tags = "")](PluginInfo(Name = "SpeedyTextureOut", Category = "EX9.Texture", Help = "Basic template which creates a texture", Tags = ""))
	#endregion PluginInfo
	public class Highspeedtextureout : DXTextureOutPluginBase, IPluginEvaluate
	{
		#region fields & pins

		[Input("Wave Phase", DefaultValue = 0.0, IsSingle = true)](Input("Wave Phase", DefaultValue = 0.0, IsSingle = true))
		IDiffSpread<float> FPhaseIn;

		[Input("Width", DefaultValue = 64, IsSingle = true)](Input("Width", DefaultValue = 64, IsSingle = true))
		IDiffSpread<int> FWidthIn;

		[Input("Height", DefaultValue = 64, IsSingle = true)](Input("Height", DefaultValue = 64, IsSingle = true))
		IDiffSpread<int> FHeightIn;

		[Import()](Import())
		ILogger FLogger;
		
		private IntPtr FData = IntPtr.Zero;

		#endregion fields & pins

		// import host and hand it to base constructor
		[ImportingConstructor()](ImportingConstructor())
		public Highspeedtextureout(IPluginHost host) : base(host)
		{
			
		}

		//called when data for any output pin is requested
		public void Evaluate(int SpreadMax)
		{
			//recreate texture if resolution was changed
			if (FWidthIn.IsChanged || FHeightIn.IsChanged) {
				//set new texture size
				this.FData =  Marshal.AllocCoTaskMem(FWidthIn[0](0) * FHeightIn[0](0) * 4);
				Reinitialize();
			}
			

			//update texture
			if (FPhaseIn.IsChanged) {
				FillData();	
				Update();
			}

		}

		//this method gets called, when Reinitialize() was called in evaluate,
		//or a graphics device asks for its data
		protected override Texture CreateTexture(Device device)
		{
			FLogger.Log(LogType.Debug, "Creating new texture...");
			return TextureUtils.CreateTexture(device, Math.Max(FWidthIn[0](0), 1), Math.Max(FHeightIn[0](0), 1));
			
		}

		//this method gets called, when Update() was called in evaluate,
		//using high speed texture fill
		//copied from a bit of vux's test code writted at Node10 (post beers)
		unsafe protected override void UpdateTexture(Texture texture)
		{
			Surface srf = texture.GetSurfaceLevel(0);
			DataRectangle rect = srf.LockRectangle(LockFlags.Discard);
			rect.Data.WriteRange(this.FData, FWidthIn[0](0) * FHeightIn[0](0) * 4);
			srf.UnlockRectangle();
		}

		private unsafe void FillData()
		{
			int w = FWidthIn[0](0);
			int h = FHeightIn[0](0);
			
			Single fw = Convert.ToSingle(w);
			Single fh = Convert.ToSingle(h);
			
			float phase = FPhaseIn[0](0);

            byte[]() byteArray = new byte[w * h * 4](w * h * 4);
            byte wave;
            
            for (int x=0; x<w; x++)
				for (int y=0; y<h; y++)
				{
					wave = Convert.ToByte(255 * 
						((Math.Sin(
						Convert.ToSingle(x)/fw *
						Convert.ToSingle(y)/fh * 10 + phase) + 1) / 2)
						);
						
					byteArray[x * 4 + y*w*4 + 0](x * 4 + y*w*4 + 0) = 255;
                    byteArray[x * 4 + y*w*4 + 1](x * 4 + y*w*4 + 1) = wave;
                    byteArray[x * 4 + y*w*4 + 2](x * 4 + y*w*4 + 2) = wave;
                    byteArray[x * 4 + y*w*4 + 3](x * 4 + y*w*4 + 3) = 255;
				}
			
			
			
            Marshal.Copy(byteArray, 0, FData, w * h * 4);
		}
	}
}

You’ll get a node called ‘SpeedyTextureOut’. And, probably, glitches every 1,2,3 seconds in the framerate. I’m not sure where it’s coming from. CPU usage isn’t very high. It’s about the same size of glitch as a recompile of the plugin.

Any ideas?

First there’s no real need for Marshal.Copy, you can write in FData Directly.

Something like:

byte* byteArray = (byte*)this.FData.ToPointer();

then access the same way

After I guess the main part comes from FillData.
I would keep all data as doubles as a starter, since .net works as good on those than on floats.

Also you can optimize some more if you use pointer instead of array:

byteArray[x * 4 + y*w*4 + 0](x * 4 + y*w*4 + 0) = 255;
byteArray[x * 4 + y*w*4 + 1](x * 4 + y*w*4 + 1) = wave;
byteArray[x * 4 + y*w*4 + 2](x * 4 + y*w*4 + 2) = wave;
byteArray[x * 4 + y*w*4 + 3](x * 4 + y*w*4 + 3) = 255;

becomes:

byteArray[0](0) = 255;
byteArray[1](1) = wave;
byteArray[2](2) = wave;
byteArray[3](3) = 255;
byteArray += 4;

If not enough will explain you some more floating point optimizations+ double buffering.

ah nice
i’m really unfamiliar with c#
so following your hints is helping out a lot

so i’ve removed the marshal.copy and changed to pointer syntax
but no change in framerate.

i’m getting 10fps at 1024x1024
if i switch out the math line:

wave = Convert.ToByte(255 * 
						((Math.Sin(
						Convert.ToSingle(x)/fw *
						Convert.ToSingle(y)/fh * 10 + phase) + 1) / 2)
						);

for a simple ‘wave= 255;’ then i get back to 60 instantly.

so it seems the maths line is the main killer.
If I replace the Math.Sin with a % 255, then i get 17fps

If i try the same code in a c++ standalone (not a balanced test) i get 30fps, and both are about 10fps with the sin.

For reference:
c#

private unsafe void FillData()
		{
			int w = FWidthIn[0](0);
			int h = FHeightIn[0](0);
			
			Single fw = Convert.ToSingle(w);
			Single fh = Convert.ToSingle(h);
			
			float phase = FPhaseIn[0](0);

            byte* byteArray = (byte*)this.FData.ToPointer();
            byte wave;
            
            for (int x=0; x<w; x++)
				for (int y=0; y<h; y++)
				{
					wave = Convert.ToByte(255 * 
						(((
						Convert.ToSingle(x)/fw *
						Convert.ToSingle(y)/fh * 10 + phase) + 1) / 2)
						% 255);
						
					byteArray[0](0) = 255;
					byteArray[1](1) = wave;
					byteArray[2](2) = wave;
					byteArray[3](3) = 255;
					byteArray += 4;
				}			
		}

c++

void testApp::update(){
	phase = ofGetElapsedTimef() * 3;

    unsigned char wave;
    float fw = w;
	float fh = h;

    for (int x=0; x<w; x++)
		for (int y=0; y<h; y++)
		{
			wave = (unsigned char)(255 * 
				((float(x)/fw * float(y)/fh * 10 + phase) + 1) / 2);
				
			colorPixels[x*4+y*4*w + 0](x*4+y*4*w + 0) = 255;
			colorPixels[x*4+y*4*w + 1](x*4+y*4*w + 1) = wave;
			colorPixels[x*4+y*4*w + 2](x*4+y*4*w + 2) = wave;
			colorPixels[x*4+y*4*w + 3](x*4+y*4*w + 3) = 255;
		}	
	
	texColor.loadData(colorPixels, w,h, GL_RGBA);

	//i could move the pointer along, then move it back again i guess
}

Anyway, all that’s a bit off topic, as my real issue is that glitch. My fps will stick at 60fps at 256x256, but every 2 seconds or so (pretty random), the whole program will pause for about 1/5s, then continue at 60fps.

I’m going to try this in a standalone plugin now as I need System.Drawing which doesn’t seem to be available in the DynamicPlugins
thanks vux!

pps, i’m not actually interested in getting that math to run quickly.
I’m actually more interested in caching images into ram to pump out as textures when needed. so there wont be any calculation at runtime.

glitch doesn’t appear in plugin from VS2008 c# ->vvvv 23. so i’m there :)

just had a look at it with a mem profiler. the reason for those glitches are garbage collections. for some reason those (which should happen very fast) gen#0 collections are somehow much more expensive in beta24 than in beta23. will have to investigate this issue a little more.

ok, investigated a little more and found two things which impact time of collection a lot in beta24. i commented that stuff out for testing and memory footprint reduced from about 34 mb down to 4mb which heavily improved the performance of those gen#0 collections. well commenting stuff out isn’t the final solution, but at least i know now what’s the problem, so thx for reporting and gn8 ;)

hi vux, since your kinect plugin is handling textures as well i guess the eratic performance behaviour seems to do with the problem described by elias. it sounds like it…

also there’s glitches in the alexp kinect driver itself on most computers.

thanks for looking into it Elias. You all back in Berlin now?

yeah i’ve seen these glitches in alexp software and vvvv. but if you find a machine which doesnt do these visible glitches then there is still a framerate problem in vvvv. and this seems to do with the elias problem.

@u7angel: Texture generation and image retrieval in the Kinect plugin is done trough unmanaged pointers, so any garbage collection here does not apply (Using CopyMemory instead of Marshal.Copy).

I think the driver is still in heavy beta stage (and not very stable, like unplugging your kinect while you reading data into it should NOT block your whole system), so i guess need to wait for driver updates more than anything.

Probably will test and move to the open source version when i can, so it’s easier with profiling/debugging.

mmm, thanks for clearing this up :)