Recently at the office we’ve been dealing with a strange BitmapData memory occupation behavior I want to share with you guys.

On this subject the AS3 reference reads “A BitmapData object contains an array of pixel data. This data can represent either a fully opaque bitmap or a transparent bitmap that contains alpha channel data. Either type of BitmapData object is stored as a buffer of 32-bit integers. Each 32-bit integer determines the properties of a single pixel in the bitmap.”

This should be meaning that a BitmapData object should be represented in memory as a sequence of 32 bits integers, one for each pixel in the bitmap, hence width*height*4 bytes which definitely makes sense.
However the actual behavior is significantly different and you can easily find out that the real BitmapData size in memory is not proportional to its area as it should be.
Let’s see an example (code shared at wonderfl)

BitmapData Memory Allocation – wonderfl build flash online

The example shows how different the size of a BitmapData is depending on its maximum dimension: the code instantiates 4096 images with an increasing area from 1 up to 4096 pixels, first with a horizontal extension (width is the increasing dimension, blue line in the chart) then with a vertical extension (height is the increasing dimension, red line in the chart).
You can notice that the behaviors of the lines in the chart are very different and the blue line’s Y is increasing only few times whilst the red one’s (vertical bitmap) is increasing much more frequently and is reaching much higher values.
So what’s the real memory representation of a BitmapData? For sure it isn’t a width*height*4 matrix as it should be and it is stricly related to the dimension the BitmapData is extended on.

We worked a bit on this thing at the office (thanx to Matteo Lanzi) and we found out that BitmapData object memory allocator has a strange behavior: it allocates N chunks of memory aligned to 256bytes (64 pixels) at time.
This would be a useful behavior if the BitmapData object would be resizable via actionscript, but it is not, or if the freed chunks were put in a memory pool and used by the next BitmapData requiring memory, but it doesn’t seem to do that ( the only case we noticed that the memory is reused is when a NxM BitmapData is created, deleted and recreated with the same N and M dimensions ).
If it was only this however it wouldn’t have been such a big problem to deal with, the real big issue is that a 256b chunk is representing a sequence of pixels with a defined horizontal orientation.
This hardcoded behavior leads to a very unpleasant side effect: a BitmapData instance with a high number of little rows (high height, little width,, hence a ‘vertical’ orientation) is occupying a HUGE AMOUNT OF MEMORY while it should be occupying the same memory of a bitmap with the same area rotated by 90°.
Let’s seesome more example just to clear out the problem:

Case 1:
- width 1px
- height 1px
- expected allocation: 4 bytes
- actual allocation: 256 bytes (minimum chunk)

a 1×1 px BitmapData allocates in the heap 1 chunk of 256 bytes while only 4 bytes would be needed!! 63x overhead!!!

Case 2:
- width 64px
- height 1px
- expected allocation: 256 bytes
- actual allocation: 256 bytes (fits exactly the minimum chunk)

the same 256 bytes are allocated by a 64×1 bitmap and the overhead disappears, what does happen with a 65×1 one?

Case 3:
- width 65px
- height 1px
- expected allocation: 260 bytes
- actual allocation: 512 bytes (2 chunks)

with a ‘max chunk size plus one’ size a 65 px wide image allocates 2 chunks, the overhead is set to approx 1.9x.

Case 4
- width 1px
- height 64px
- expected allocation: 256 bytes (same area of the 64×1 one…)
- actual allocation: 16 Kb!!!!!

this is the worst case, every in 1xN bitmaps have the maximum overhead possible: 63x!
So try to think about an image very narrow and high, let’s say a 1×4096 one: you would expect it to be 16kb in memory (4096*4 bytes) but it actually is 1 whole megabyte (4096*4*256!!)!!

So, the formula to calculate the ACTUAL memory allocated by a BitmapData expressed in bytes I came up with is:

Math.max( 1 , height*(width/64)>>0 )*256

You can see a more detailed and complete dataset in this google spreadsheet http://bit.ly/FlashBitmapDataAllocation (feel free to spread the link) containing data for BitmapData memory occupation from 1×1 to 1×4096 and from 1×1 to 4096×1.
The test to reproduce the dataset is available on github at this url https://github.com/pigiuz/Tests/blob/master/src/BitmapDataMemoryAllocation.as (feel free to share this link as well)

This behavior has been tested in AIR 3.1 for Mac OSX 10.6 (snow leopard), 10.7 (lion) (thanx to Marco Nava for testing on lion), Window 7, Android 4.0, iOS 5 (thanx to Shawn Blais for testing on mobile),
Flash Player 11,1 by yourself (if you ran the swf linked backward in the post and noticed the same behavior).

After figuring out the problem we opened a bug in adobe’s jira, if you have comments which can raise the attention of adobe on this stuff or add something to the discussion please feel free to comment at this url https://bugbase.adobe.com/index.cfm?event=bug&id=3094186 (you’re welcome to comment here too btw )

Few more points worth some attention:
I tried to measure the memory in 3 ways:
- the data got back from flash.sampler.getSize() was not accurate, most of the time it was totally wrong and not predictable (or perhaps I didn’t get how it works in relation with BitmapData objects)
- the delta calculated with a pre and post sampling of System.privateMemory is more accurate than getSize() and fits the expected results in AIR but it’s not as accurate as I expected: in the spreadsheet linked backwards you can see that the results have a coherent trend, but there’s a lot of garbage data and little bitmaps weren’t detected at all.
- the delta calculated witn a pre and post sampling of System.totalMemory is more accurate than System.privateMemory only in flash player.

That said,
I find this allocation behavior weird, and the worst thing is that it UNDOCUMENTED and the docs are saying a different thing. I see one VERY significant bad side effects in this way of allocating bitmaps memory that involve how BitmapData objects are stricly tied to the Flash Player’s and AIR runtime’s APIs:
flash.display.Loader generates BitmapData instances when it loads images from URL or streams, so if an application needs to load many bitmaps that don’t fit the memory alignment there will possibly be a HUGE memory waste.
A possible solution could be the creation of few additional functions such as

loadInto(urlrequest:URLRequest, destination:ByteArray):void

and

loadBytesInto(bytes:ByteArray,destination:ByteArray):void

which both take a data source and a preallocated bytearray as input and put outputs to the preallocated bytearray called destination.

This is the first shot, I’d really like you guys to spread the word and help me to make adobe sensible on this topic as it’s really worth some effort to make flash and air runtimes even better than what they are now.

Thank you all in advance,
ciao,

pigiuz

Short tips post

BitmapData size limit:
Up to flash 9 the size limit for a BitmapData object was 4096×4096 pixels.
With flash player 10 that limit was removed, but what does this exactly mean? May we be able to create 4097×4097 sized bitmapdata instances? the answer is NO, we can’t.
I just found an official explanation of that here http://kb2.adobe.com/cps/496/cpsid_49662.html

Just in case you don’t have the time\will to read it:
- we still have a limit!
- the limit is set to the maximum amount of pixels (16,777,215 (the decimal equivalent of 0xFFFFFF))
- the maximum valid size of the bounding rectangle side is 8191

Stage size limit:
It actually depends on the stage quality.
With a high quality set the bound limit is 4050×4050, if your content exceeds it gets cropped.
With a low or medium quality the bound limit increases, there’s no official documentation about that (or at least I didn’t find it).
Note that Adobe’s saying that “graphic artifacts” could be displayed when our stage “approaches” 3840 pixels range.

have fun

I’m pleased to be speaking at “Arrivano i guru” stage of Milan the next September 26th.
Arrivano i guru is a moving conference iterating the whole italian peninsula. Here’s the complete list of stages
Main topics are Adobe Flash Platform , SEO, and graphics. Great speakers are involved, such as Tiziano Fruet, Luca Mezzalira, and many others.

This is my session’s outline:
(ITALIAN)
Special Effects in Flash CS4
Impariamo ad utilizzare bitmaps, filtri e tweens in Flash CS4 per creare effetti speciali custom.

I filtri in flash
- cosa sono i filtri in flash
- applicare i filtri con AS3

I blendmode in flash
- cosa sono i blendmode in flash
- applicare i blendmode AS3

Sveliamo i bitmap
- cos’è un bitmap in Flash
- i bitmap come tela di un pittore
- manipolare i bitmap in AS3

TweensyFX
- cos’è Tweensy
- animare i propri effetti speciali

(ENGLISH)
Special Effects in Flash CS4
Let’s learn to use bitmaps, filters and tweens within Flash CS4 to create custom special effects.

filters
- what are filters in flash?
- applying filters using AS3

blendmodes
- what blend modes are
- applying blendmodes using AS3

discovering bitmaps
- what are bitmaps for Flash?
- bitmaps as a painter’s canvas
- manipulating bitmaps using AS3

TweensyFX
- what’s Tweensy?
- creating and animating our own special effects

See you there

Yesterday night it was too warm to sleep and I have no air conditioning at home, so I thought it was a good opportunity to make some experiments related to the high temperature… “the burning man”

Here’s the algorithm in human language:
- pick up the image from the webcam
- perform a threshold on it to exclude (alpha 0) every pixel below the average skin amount of red (between 44 and 90 depending on the light)
- perform a second threshold to exclude (alpha 0) every pixel above the average skin amount of green and blue (between 3344 and 6677 depending on the light)
- draw the result onto a new bitmapdata with the ADD blendmode
- apply blur to the bitmapdata
- generate a perlin noise of one channel (i chose red)
- displace the bitmapdata towards -y
- apply again a displacement map filter to displace horizontally (i used a cos function to make the x displacement a little likely)
- apply a displacement map towards +y with just a bit less strenght than the previous “-y” one (this is to make the fire propagation much more likely)
- then merge the original webcam output with the displaced one…and you’ve the burning man!

here the actionscript implementation

package
{
	import flash.display.Bitmap;
	import flash.display.BitmapData;
	import flash.display.BitmapDataChannel;
	import flash.display.BlendMode;
	import flash.display.Sprite;
	import flash.events.Event;
	import flash.filters.BlurFilter;
	import flash.filters.DisplacementMapFilter;
	import flash.filters.DisplacementMapFilterMode;
	import flash.geom.ColorTransform;
	import flash.geom.Matrix;
	import flash.geom.Point;
	import flash.geom.Rectangle;
	import flash.media.Camera;
	import flash.media.Video;

	[SWF(width="320",height="240",frameRate="31",backgroundColor="0x000000")]
	public class BurningMan extends Sprite
	{

		private var _cam:Camera;
		private var _vid:Video;

		private var _fireThreshold:BitmapData;
		private var _firePerlin:BitmapData;
		private var _fireOffset:Point;
		private var _fireSpeed:int = 20;
		private var _fireDisplaceUP:DisplacementMapFilter;
		private var _fireDisplaceDWN:DisplacementMapFilter;
		private var _fireDisplaceSrc:BitmapData;
		private var _fireBlur:BlurFilter;
		private var _fireColorTransform:ColorTransform;

		private var _fireMerge:BitmapData;

		private var _camBd:BitmapData;
		private var _b:Bitmap;

		private static const WIDTH:Number = 320;
		private static const HEIGHT:Number = 240;
		private static const ORIGIN:Point = new Point();
		private static const REFLECT:Matrix = new Matrix(-1,0,0,1,WIDTH,0);
		private static const RESIZE_2X:Matrix = new Matrix(2,0,0,2);

		private var _rect:Rectangle;

		public function BurningMan()
		{
			super();
			init();
		}

		private function init():void
		{
			_cam = Camera.getCamera();
			_cam.setMode(WIDTH,HEIGHT,31);
			_vid = new Video(WIDTH,HEIGHT);
			_vid.attachCamera(_cam);

			_fireOffset = new Point();
			_fireThreshold = new BitmapData(WIDTH,HEIGHT,true,0x00000000);
			_fireMerge = new BitmapData(WIDTH,HEIGHT,true,0x00000000);

			_firePerlin = new BitmapData(WIDTH*.5,HEIGHT*.5,false,0x000000);
			_fireDisplaceSrc = new BitmapData(WIDTH,HEIGHT,false,0x000000);
			_fireDisplaceUP = new DisplacementMapFilter(_fireDisplaceSrc,ORIGIN,BitmapDataChannel.RED,BitmapDataChannel.RED,0,-15,DisplacementMapFilterMode.CLAMP);
			_fireDisplaceDWN = new DisplacementMapFilter(_fireDisplaceSrc,ORIGIN,BitmapDataChannel.RED,BitmapDataChannel.RED,0,9,DisplacementMapFilterMode.CLAMP);
			_fireBlur = new BlurFilter(2,2,4);
			_fireColorTransform = new ColorTransform(1.3,1.1,1,.7);

			_camBd = new BitmapData(WIDTH,HEIGHT,true,0x00000000);
			_rect = _camBd.rect;
			_b = new Bitmap(_camBd);
			_firePerlin.lock();
			_fireMerge.lock();
			addChild(_b);

			addEventListener(Event.ENTER_FRAME,update);
		}

		private function update(e:Event=null):void
		{
			//locks
			_camBd.lock();

			_fireOffset.y+=_fireSpeed;
			_firePerlin.perlinNoise(_firePerlin.width*.125,_firePerlin.height*.125,1,0,true,false,BitmapDataChannel.RED,false,[_fireOffset])
			_fireDisplaceSrc.draw(_firePerlin,RESIZE_2X);

			_camBd.draw(_vid,REFLECT);
			_fireThreshold.setVector(_rect,_camBd.getVector(_rect));
			_fireThreshold.threshold(_fireThreshold,_rect,ORIGIN,"<",0xFF440000,0x00000000,0xFF0000,true);
			_fireThreshold.threshold(_fireThreshold,_rect,ORIGIN,">",0xFF003344,0x00000000,0x00FFFF,true);
			_fireMerge.draw(_fireThreshold,null,_fireColorTransform,BlendMode.ADD,_rect);

			_fireDisplaceUP.scaleX = Math.cos(((_fireOffset.y/(2<<3))%360)*180/Math.PI)*2;
			_fireDisplaceDWN.scaleX = -_fireDisplaceUP.scaleX;
			_fireMerge.applyFilter(_fireMerge,_fireMerge.rect,ORIGIN,_fireBlur);
			_fireMerge.applyFilter(_fireMerge,_rect,ORIGIN,_fireDisplaceUP);
			_fireMerge.applyFilter(_fireMerge,_rect,ORIGIN,_fireDisplaceDWN);
			_camBd.draw(_fireMerge,null,_fireColorTransform,BlendMode.HARDLIGHT,_rect);

			//unlocks
			_camBd.unlock();
		}
	}
}

here's a preview of myself empowered by the force of the fire!:D

click the photo to see the demo in action

Today I encountered this blog post from Zevan (which blog is REALLY a good daily reading I suggest everyone to take) about bitmapData merging and I started tweaking some code doing some benchmarks to find out which way is the most performing. Here are my tests:

First strike (Zevan’s one): copyPixels

[SWF(width=650, height=650)]
var loader:Loader = new Loader();
loader.load(new URLRequest("http://actionsnippet.com/wp-content/chair.jpg"));
loader.contentLoaderInfo.addEventListener(Event.COMPLETE, onLoaded);
var w:Number;
var h:Number;
var rows:Number = 20;
var cols:Number = 20;
var tiles:Vector. = new Vector.();
var locX:Vector. = new Vector.();
var locY:Vector. = new Vector.();
var rX:Vector. = new Vector.();
var rY:Vector. = new Vector.();
var sX:Vector. = new Vector.();
var sY:Vector. = new Vector.();
function onLoaded(evt:Event):void{
	w = evt.target.width;
	h = evt.target.height;
	var image:BitmapData = Bitmap(evt.target.content).bitmapData;
	var tileWidth:Number = w / cols;
	var tileHeight:Number = h / rows;
	var inc:int = 0;
	var pnt:Point = new Point();
	var rect:Rectangle = new Rectangle(0,0,tileWidth,tileHeight);
	var startTime:Number = getTimer();
	for (var i:int = 0; i
in my machine (mbp, osx) it takes ~27 ms to extract data and ~2-3ms each iteration for setting data on the canvas bitmapData
Second strike: getVector\setVector
[SWF(width=650, height=650)]
var loader:Loader = new Loader();
loader.load(new URLRequest("http://actionsnippet.com/wp-content/chair.jpg"));
loader.contentLoaderInfo.addEventListener(Event.COMPLETE, onLoaded);
var w:Number;
var h:Number;
var rows:Number = 20;
var cols:Number = 20;
var tiles:Vector.> = new Vector.>();
var tileRect:Rectangle;
var locX:Vector. = new Vector.();
var locY:Vector. = new Vector.();
var rX:Vector. = new Vector.();
var rY:Vector. = new Vector.();
var sX:Vector. = new Vector.();
var sY:Vector. = new Vector.();
function onLoaded(evt:Event):void{
	w = evt.target.width;
	h = evt.target.height;
	var image:BitmapData = Bitmap(evt.target.content).bitmapData;
	var tileWidth:Number = w / cols;
	var tileHeight:Number = h / rows;
	tileRect = new Rectangle(0,0,tileWidth,tileHeight);
	var inc:int = 0;
	var pnt:Point = new Point();
	var rect:Rectangle = new Rectangle(0,0,tileWidth,tileHeight);
	var startTime:Number = getTimer();
	for (var i:int = 0; i;
	  var startTime:Number = getTimer();
	  for (var i:int = 0; i
on my machine it takes only ~9 ms to extract tiles slices (yes, 3 times faster!!!) and ~1ms for pushing them all in the canvas for each loop iteration
Third strike: getPixels\setPixels
[SWF(width=650, height=650)]
var loader:Loader = new Loader();
loader.load(new URLRequest("http://actionsnippet.com/wp-content/chair.jpg"));
loader.contentLoaderInfo.addEventListener(Event.COMPLETE, onLoaded);
var w:Number;
var h:Number;
var rows:Number = 20;
var cols:Number = 20;
var tiles:Vector. = new Vector.();
var tileRect:Rectangle;
var locX:Vector. = new Vector.();
var locY:Vector. = new Vector.();
var rX:Vector. = new Vector.();
var rY:Vector. = new Vector.();
var sX:Vector. = new Vector.();
var sY:Vector. = new Vector.();
function onLoaded(evt:Event):void{
	w = evt.target.width;
	h = evt.target.height;
	var image:BitmapData = Bitmap(evt.target.content).bitmapData;
	var tileWidth:Number = w / cols;
	var tileHeight:Number = h / rows;
	tileRect = new Rectangle(0,0,tileWidth,tileHeight);
	var inc:int = 0;
	var pnt:Point = new Point();
	var rect:Rectangle = new Rectangle(0,0,tileWidth,tileHeight);
	var startTime:Number = getTimer();
	for (var i:int = 0; i
a nice one, ~ 17 ms to extract and ~2ms to loop on my mac, faster than copyPixels but vectors are still leading...
Fourth strike: merge
[SWF(width=650, height=650)]
var loader:Loader = new Loader();
loader.load(new URLRequest("http://actionsnippet.com/wp-content/chair.jpg"));
loader.contentLoaderInfo.addEventListener(Event.COMPLETE, onLoaded);
var w:Number;
var h:Number;
var rows:Number = 20;
var cols:Number = 20;
var tiles:Vector. = new Vector.();
var locX:Vector. = new Vector.();
var locY:Vector. = new Vector.();
var rX:Vector. = new Vector.();
var rY:Vector. = new Vector.();
var sX:Vector. = new Vector.();
var sY:Vector. = new Vector.();
function onLoaded(evt:Event):void{
	w = evt.target.width;
	h = evt.target.height;
	var image:BitmapData = Bitmap(evt.target.content).bitmapData;
	var tileWidth:Number = w / cols;
	var tileHeight:Number = h / rows;
	var inc:int = 0;
	var pnt:Point = new Point();
	var rect:Rectangle = new Rectangle(0,0,tileWidth,tileHeight);
	var startTime:Number = getTimer();
	for (var i:int = 0; i
~36 ms to extract and ~12ms to merge tiles on the canvas!!!...too slow all the way...:\
There are still some methods left such as getPixel\setPixel, getPixel32\setPixel32, and copyChannel but they're a too much restrictive choice because they're handling one pixel, or channel at time therefore a further loop would be required to get them doing this task.
Summary:

getVector\setVector : ~9ms\~1ms

getPixels\setPixels: ~17ms\~2ms

copyPixels: ~27ms\~2-3ms

merge: ~36ms\~12ms
make your choice  
NOTE: these benchmarks are valid from flash player 10 because we (both me and Zevan) used the Vector native type to store lists of typed data. To make them valid for previous version of the player make sure to replace vectors with arrays and check other types are already supported by the target player.
stay tuned