If you are going to the event next week in Berlin then let me know about it. Maybe we can meet face to face and if there are enough of us perhaps even a gourp community meeting. This can be a good opportunity to meet the experts.

In any case, you are all welcome to join my session titled "Parallel Programming for Embedded". I will be presenting on Friday 10:45 - 12:00.

At the basis of this presentation is the fact that the hardware has always been parallel. This also caused the kernel drivers to live in a parallel environment, so even though embedded devices were late to adopt Multi-Core CPUs, the people who are working with the lower levels have always been working in parallel environments.

The session speaks of parallel systems in general side by side with embedded systems and infrastructure environemnts.

The goal of this session is to open the eyes and show the systems that have always been working in parallel and name the principles used with these systems.

You can read my previous blogs to learn more about this approach. For example these:

is dos the ideal parallel environment part iv

stateful programming a case study

Here are a few slides from this presentation.

Parallel Programming for Embedded TechEd 2009

USB Ping Pong

Hope to see you all there,
Asaf

This time my attention was attracted by a message in RSDN "Alignment on 64-bit architectures" running as follows:

Today I have faced a problem in Linux. There is a data structure consisting of several fields: 64-bit double, 8 unsigned char and one 32-bit int. Altogether it is 20 bytes (8 + 8*1 + 4). On 32-bit systems sizeof is 20 bytes and everything is OK. But on the 64-bit Linux sizeof returns 24 bytes. That is, an alignment at the 64-bit border takes place.

After that the author dwells upon data compatibility and asks for advice how to pack data in the structure. But at the moment we are not interested in this. What we are interested in is that there is a new type of errors which can occur when porting applications on a 64-bit system.

It is clear and common that when sizes of fields in a structure change, the size of the structure itself changes too because of this. But this is a different case. The size of the fields remains the same but the size of the structure will change too due to different alignment rules. This behavior can lead to various errors, for example, incompatibility of the formats of the data being saved.

Linux systems are not supported by Viva64 yet, but I decided to find out if such an error can occur in Windows systems. For this purpose I took an example of the code printing types' sizes and alignment from the article "C++ data alignment and portability". I've modified it a bit for Visual Studio and got this program:

#include <iostream>
using namespace std;

template <typename T>
void print (char const* name)
{
  cerr << name
       << " sizeof = " << sizeof (T)
       << " alignof = " << __alignof (T)
       << endl;
}

int _tmain(int, _TCHAR *[])
{
  print<bool>        ("bool          ");
  print<wchar_t>     ("wchar_t       ");
  print<short>       ("short int     ");
  print<int>         ("int           ");
  print<long>        ("long int      ");
  print<long long>   ("long long int ");
  print<float>       ("float         ");
  print<double>      ("double        ");
  print<long double> ("long double   ");
  print<void*>       ("void*         ");
}

I compared the data I'd got with the data described in the article "C++ data alignment and portability" for GNU/Linux systems and now give them in Table 1.

Table 1. Types' sizes and alignment.

Let's study this table. Pay attention to the marked cells relating to long int and double. These types' sizes don't depend on the architecture's size and therefore don't change. Both on 32-bit and 64-bit systems their size is 8 byte. But alignment is different for 32-bit and 64-bit systems. It can cause change of the structure's size. When we implement Viva64 for Linux we'll take into consideration the possible errors relating to this.

In Windows systems, there are no such problems with alignment change. Pay attention that alignment of all the types doesn't change or changes together with the type's size. That is good - Windows developers have one potential problem off.

You are hereby invited to a gathering on November 17 at 5:30 pm in room C124 of the Oregon Convention Center in Portland. We are having a panel discussion focused on incorporating parallelism into the Computer Science curriculum. Of course, you might need to purchase a day pass to SC09 to attend, if you are not otherwise attending.

We generally all believe it is a good idea to weave parallelism throughout the CS curriculum. There are a variety of paths to get there, roughly corresponding to number of panelists, of which I will be the moderator and facilitator. The panel held last year at SC08, had a standing-room-only animated crowd. It happens to be available in 15 minute chunks: part 1, 2, 3, 4, 5, 6, and 7. I expect similar festivities this year.

Our distinguished panel is composed of

• Wen-mei Hwu, University of Illinois at Urbana-Champaign, who believes key computer architecture concepts, some performance optimization concepts, some program analysis concepts, some program transformation concepts, effective parallel programming patterns and algorithms, and fundamental parallelism concepts are required to train a successful undergraduate CS major.
• James Larus, Microsoft Research, who believes parallelism must be woven into the general series of courses, but sees the dilemma of the current awkward and crude tools imposing awareness of the underlying architecture on the student. He longs for a more elegant toolchain for use with students.
• Simon McIntosh-Smith, University of Bristol, who has succumbed to the "Snow White" syndrome of advocating the dwarves parallel patterns from UCB. He also sees the utility of teaching the current prevalent tools: MPI and OpenMP, while remaining sensitive to the emerging potential standards: CUDA, OpenCL, Ct, map-reduce, etc.
• Bob Chesebrough, Intel, who sees the trouble in River City of a late introduction of parallelism to students, and of course recommends the "Think Method", in this case the "Think Parallel" method. Parallelism is all around and the key concepts can be conveyed without a computer.
• Steve Parker, nVidia, who believes and has done many things. There needs to be a teaser to get you to come to the event. This is it.
• Charlie Peck, Earlham College, who sees faculty education as key activity to get right, since many CS faculty have limited first hand experience with the myriad faces of parallelism. Involving students is a key part of this, for a unique form of learning takes place via an apprenticeship model of student involvement.

For several years now, we have been a parallel education working group composed of broad representation from industry, colleges/universities, hardware manufactures, and labs We are about to call ourselves something like "The Educational Alliance for a Parallel Future." I now know first hand what is involved in morphing from a working group to an alliance. When you are getting ready to make a tee-shirt, alliance ends up sounding sounding way cooler, and actually better reflects who we are. We will also be releasing a parallel crossword puzzle. This is not one solved in parallel, though it might be if someone is looking over your shoulder. The subject matter is of many things parallel. Stay tuned, we'll point you to it, once it debuts at SC09.

After you look at the technology, think about the kinds of device capabilities, hardware and software, that would make your web application better. Then share them with us - we're listening.
If you need some food for thought take a look at this blog post - where I list several API drafts we recently submitted to the W3C Device API working group.

"The story" gives you the context. "The Technology" dives a little deeper into the details.

Working on CAD Exchanger I am designing one of its plugin to convert 3D CAD data between ACIS and Open CASCADE (two modeling kernels) to be parallel. Depending on a model size, the converter has to deal with multiple small objects allocated on a heap (e.g. 20,000+ objects each taking 48bytes + additional object data such as lists, strings, etc).

The translation works just fine and concurrency analysis with Intel Parallel Amplifier indicates high concurrency levels. So far, so good. However I had noticed that when translating the same ACIS file over and over again in the same test harness session translation took longer and longer. Why could it be ?

So I launched the Amplifier to collect hotspots and here is what I saw:

These two top hotspots relate to the memory manager layer (Standard_MMgrRaw class) which simply forwards calls to malloc/free and new/delete. Trying to root-cause the problem I had to switch to the mode to see direct OS functions (toggling off the button on the Amplifier toolbar) and here is a new screenshot:

It shows that hotspots are two system functions – RtlpFindAndCommitPages() and ZwWaitForSingleObject() – which are called from memory allocation / deallocation routines. It also shows that the nearest hotspot related to my code (BSplCLib::Bohm()) is just 1/4 of the time consumed by ZwWaitForSingleObject() (0.47s vs 1.81s).

After experimenting with several runs, analyzing how the hotspot profile changes with growing number of runs, I concluded that the first hotspot is explained by the fact that the ACIS converter creates multiple tiny objects with different size with short life span (they are destroyed after every conversion). This seems to cause strong memory fragmentation which forces the system to constantly look for new memory chunks.

The second hotspot (ZwWaitForSingleObject()) which goes through critical section is caused by the default mechanism of memory management on Windows which uses a lock.

The execution of locks&waits analysis also proves that memory management lock is the greatest one adversely affecting concurrency.

All this is caused by the direct use of calloc/malloc/free, and new/delete called dozens of thousands times. It’s worth mentioning that such hotspots did not exist when I used serial implementation and popped up only when I started using parallel one. The former used a memory manager (in a 3rd party lib) that allocated memory blocks and did not return them to the system reusing them when the application requested new blocks. I couldn’t reuse this memory manager because it was not thread-safe and therefore had to switch to another manager that simply forwarded to malloc/free.

So I almost was forced to write my own memory manager that would implement a previous behavior and would be thread-safe and … fast ! Challenges are good but not when you need to re-write low-level components what can take a lot of time and require diligent thorough testing delaying progress in your project which already receives very limited attention.

So, I approached my colleagues from the Threading Build Blocks team to check if there is anything TBB could help with. What was my surprise when they suggested me trying a new release 2.2. Version 2.2 offers a mechanism to seamlessly replace the system memory manager with the tbb allocator. ‘Seamlessly’ really means it – everything I had to is to add a single line of code into a C++ file:

#include "tbb/tbbmalloc_proxy.h"

The outcome was immediate. Not only did the hotspot profile change completely removing the OS hotspots (see the comparison mode screenshot below) but the overall speed up (on entire test case) was about 25%! One line of code, no need of re-writing anything on my own saved hours of coding, with such a return! Just incredible, the least I could say. Recently released 2.2 Update 1 includes further improvements which my app now benefits from (more reliable processing of realloc(), bug fixes for debug mode, etc).

The colleagues later explained me that the TBB allocator runs concurrently (seemingly without any locks inside) and with a similar fashion of reusing previously allocated blocks. Thus, it was the entire application (not only its parallel part) which benefited from this substitution.

So, if you are migrating from serial to parallel implementation you may encounter something unexpected – memory bottlenecks. If you got accustomed to use some nice single-threaded memory manager you can be forced to consider migration to something alternative. If this is the case you may want to give a try to tbb allocator and see if it helps in your case.

12/3 | 4PM | 2405 Siebel Center for Computer Science

UPCRC Illinois Research Seminar: A Structured, Unified Approach to Multi-Core and Many-Core Computing - with Applications by Michael McCool, Intel & U of Waterloo

In a previous post I kicked off a conversation with web developers about exposing device capabilities as javascript APIs and widgets in the browser.

Here's your chance to comment and help inform our web developer enabling activities. Tell us about how having better access to the device's hardware capabilities would make your web app implementation easier. Maybe you have a use case you would like to share?

Here is a list of APIs for device capabilities we proposed.

You can see the detailed API specs and example use cases at the WC3's site.
This is only the beginning. Stay tuned to the blog, or my tweets, for more API, widget and mashup proposals coming in the near future.

"Debugging bad pointer casts in 64bits"
I am currently converting a program to Windows 64bits. Some bad code casts
pointers into long and vice-versa.
Code example:

MyObj* pObj = …
  ::SendMessage(hwnd, msg, (WORD)x, (DWORD)pObj);

The problem is that it is impossible to detect such errors with the help of the compiler as explicit type conversion is used which suppresses diagnostic messages. The static code analyzer Viva64 is used for diagnosing these and many other errors.

Viva64 is a specialized commercial tool for searching errors in C/C++ programs when porting them on 64-bit systems or developing new 64-bit programs. Besides, the tool allows you to better optimize 64-bit code. The tool is rather expensive but our users can get support in setting and improving the tool according to the peculiarities of their projects what makes the tool very effective.

This tool serves for detecting explicit type conversions which are dangerous from the viewpoint of a 64-bit data model.

For example, on the code given above the analyzer will show warning V202:
V202. Explicit type conversion. Type casting from memsize to 32-bit.

And on the following code where incorrect explicit type conversion from a 32-bit to a 64-bit type is used:

unsigned width, height, depth;
DWORD_PTR arraySize = DWORD_PTR (width * height * depth);

you'll get warning V201:
V201. Explicit type conversion. Type casting to memsize.

Conclusion: if you have a large code it is more reasonable not to perform eternal testing catching new overflows/cut of values at the next set of input data but to buy Viva64 and catch these errors at the very early stages.

The analyzer considers the last fragment of the code suspicious, as it is very similar to a typical error. The fact is, that if width, height and depth meanings are quite large, during multiplication, 32-bit type unsigned overflow can occur. And this incorrect result will be expanded to 64-bit DWORD_PTR type and placed in arraySize variable. There is high probability that the code should be rewritten in the following way:

unsigned width, height, depth;
DWORD_PTR arraySize = DWORD_PTR (width) *
  DWORD_PTR (height) * DWORD_PTR (depth);

In this case, 64-bit types will be multiplied, and the overflow will disappear. Such situations are described in detail in the article "20 Issues of Porting C++ Code on the 64-bit Platform".

Additional links:

• Viva64: what is it and for whom is it meant?
  http://www.viva64.com/art-1-2-903037923.html
• 64 bits, Wp64, Visual Studio 2008, Viva64 and all the rest…
  http://www.viva64.com/art-1-2-621693540.html

Silverlight 3.0 development is supported within Visual Studio 2010 and it includes the Silverlight 3 developer runtime and Silverlight 3 SDK build 3.0.40818.

Silverlight 3 tools for Visual Studio 2008 and Expression Blend 3 included SDK build 3.0.40624 and if these are installed when you run the Visual Studio 2010 beta 2 installer an error will be reported. Fear not however and refer to the blog post by Amy Dollard on MSDN.

We have only 24 hours left until the General Availability (GA) of Windows 7 and so tomorrow Windows 7 will be in the hands of consumers for the first time. With the release of Windows 7, along with the release of some awesome new processors from Intel, it was time for a new personal notebook and also for a companion netbook as Windows 7 has been designed to work great on netbooks.

If you've been following this blog for sometime you'll know that I use Hyper-V for virtualization within Windows Server 2008 R2, the server variant of Windows 7, with this in mind I found the HP Envy 15. Inside this machine is the mobile Core i7 processor which is quad-core and supports hyper-threading allowing the execution of 8 concurrent threads, and for virtualization there is 16Gb of DDR3 memory.

I also ordered the HP Mini 311 which is powered by an Intel Atom processor as a companion device that can be used when travelling. I chose this netbook over others on the market because you can install 3Gb of DDR3 memory. 

I'll be blogging next month about the experience of Windows 7 on both the Core i7 and Atom processors, specifically with the software developer in mind.

It should also be mentioned that this blog post does not represent an endoresemnt, by Intel Corporation, of the particular laptop and netbook models shown above. I purchased these machines based on my own research and my own requirements.

Windows 7 is an outstanding operating system (imho) and I'd be very interested to hear what you think of the operating system and the Core i7 and Atom processors.

Yesterday beta 2 of Visual Studio 2010 was released and Microsoft confirmed that the official launch date for Visual Studio 2010 will be March 22, 2010.

After downloading the build one notable absentee was RIA Services although according to Brad Abrams the support for RIA services will be available at the PDC next month.

Bjarne Stroustrup, creator and champion of the C++ language, is among other things a collector of aphorisms, and this is one of his favorites. It’s a deep idea, but I want to touch on just one aspect of this surprising equivalence, and tie it to, of all things, the Intel Optimization Reference.

The commonality I’m thinking of is the concept of deprecation. One thing that languages and libraries (or APIs if you prefer) share is that they are snapshots of what seems to be the best possible set of tools and concepts for solving problems in a specific domain. There’s an art of doing this well, because it requires thinking deeply with imperfect knowledge of the future. So, no library (or language) is perfect. All libraries can have features with flaws, and when a flaw becomes obvious enough, it can be addressed with some alternative feature that provides the functionality of the flawed original.

A classic example from the C runtime library are the string functions, such as strcpy(). strcpy() works on null-terminated strings of arbitrary length. This seemed simple, performant, and straightforward at the time the first C runtime libraries were created, but the lack of a “safety net” if a null terminator was omitted made it not just a painful source of errors, but also an ally of malicious coders. If a virus writer could guess you were using strcpy() behind a secure API, he or she was one step closer to a successful buffer overrun exploit.

So, strncpy() was created. It provides a safety net in the form a length parameter that the copy operation won’t exceed. All strcpy() calls should be replaced by strncpy() because of this advantage. But, the vast existing codebase cannot be cannot be changed overnight.

So, deprecation is the answer. It’s a policy that depends on developer compliance for success. New code should always use the new method (e.g., strncpy()) existing code should be updated whenever possible. Tools like compilers may be adapted to warn about the continuing presence of deprecated functions like strcpy(), but those functions will continue to work as designed.

In many cases, a large application may be updated to remove flawed APIs only in stages, over time. If a function or module is being updated for some other purpose, a new feature for example, a careful developer will take the opportunity to remove or replace deprecated library calls.

Languages too, can have deprecated features. And in the case of an assembly language, the most interesting of those “features” are typically instructions or instruction sequences that should be avoided for performance reasons. In this sense, the language is really the Instruction Set Architecture itself. And this is where the Intel Optimization guide comes in. For example, Assembly/Compiler Coding Rules 28-31 from the Optimization Reference address the issue of instruction selection, and Rule 31 calls out by name instructions like Enter and Loop that should be avoided altogether.

Very few developers are creating new assembly routines anymore. Compiler quality and intrinsics support have greatly reduced the need for new assembly code. But there is a significant existing codebase out there. An existing assembly function may be opened up for changes, but not wholesale replacement. I have been in this situation several times in my career. If you are a developer working on such a function, the assembly coding rules in the optimization are your friend.

Just as you might replace deprecated library calls in a C function you are updating, the assembly coding rules can help you identify “deprecated” assembly code instructions that can be replaced with straightforward equivalents that will have better performance on current CPUs and those in the foreseeable future. And just as a development tool can call attention to deprecated APIs, performance analysis tools such as Intel VTune can flag assembly instructions that should be avoided.

Intel doesn’t technically call such instructions “deprecated”. In part that’s because deprecation implies eventual removal, and that’s far more problematic prospect for an ISA than for most libraries. But the principle is the same, and I hope that the parallels are compelling and clarifying.

My next few posts will call out some specific assembly coding rules from the optimization guide, and how those rules fit in with this analogy to API deprecation.

In the meantime, ponder the common sentiment shared by library designers, language designers, and instruction set architects when they reflect on those features they created which did not age gracefully, “It seemed like a good idea at the time”.

Microsoft today released beta 2 of Visual Studio 2010 and confirmed that the official launch date for Visual Studio 2010 will be March 22, 2010.

Visual Studio 2010 beta 2 was uploaded to MSDN subscriber downloads shortly after 9am PST this morning and this build includes a "go-live" license. You can read more about the release on Somasegar's blog here.

Visual Studio Team Suite has been renamed Visual Studio 2010 Ultimate and the product also includes a new logo that is very similar to the mathematical infiniti symbol.

MSDN also seems to have a new logo too...

Once opened, the Visual Studio 2010 environment can be seen here with the revised New Project dialog.

If you're interested in the build numbers of Visual Studio 2010 and the associated .net framework here is the about dialog.

Over the next few days and weeks I'll post extensively about Visual Studio 2010 and Team Foundation Server 2010.

It began with a complaint that the static analyzer Viva64 hangs when analyzing one project, or rather one file of the project. Analysis showed that the analyzer doesn’t hang at all and on the contrary works very hard. Very hard and very long. I think even this analyzer can analyze it successfully in approximately 5 or 10 hours :). But this became clear later and of course such behavior should be considered an error.

Frankly speaking, at the beginning I was astonished by this behavior. Viva64 analyzer operates rather quickly, and average time of analyzing one file takes just a few seconds. And its operating time is much less than time needed for preprocessing files. Preprocessing is executed by Visual C++ compiler and we cannot speed up this process yet. That’s why, roughly speaking, the time of testing a project is close to the time of preprocessing all the files. And suddenly we discover such a great ineffectiveness in the analysis algorithm!

Investigations showed that hanging occurs when processing the file which contains some resources from Qt library, more exactly array whose size… well, I don’t know its size, but I know that this is the largest array I’ve ever seen. It looks as follows:

static const unsigned char qt_resource_data[] = {
  0x0,0x0,0x0,0xc5,
  0x51,
  0x46,0x72,0x61,0x6d,0x65,0x20,0x7b,
  0xd,0xa,0x20,0x20,0x20,0x20,0x6d,0x69,0x6e,
  0x2d,0x77,0x69,0x64,0x74,0x68,0x3a,0x20,
  0x36,0x70,0x78,0x3b,0xd,0xa,0x20,0x20,
  0x20,0x20,0x6d,0x69,0x6e,0x2d,0x68,0x65,
  0x69,0x67,0x68,0x74,0x3a,0x20,0x36,0x30,
  . . .

This array occupies about 9 Mb in the file. This is a lot but it’s not reason for the analyzer for such a bad behavior.

The problem was the already mentioned last() function. Viva64 analyzer is built on VivaCore library. And VivaCore is built in its turn on OpenC++ library from which it inherited data representation as trees and lists. By the way, data processing in OpenC++ (and VivaCore as well) reminds a lisp program very much. There are a lot of functions of the type like Eq, Car, Cdr, First, Rest, Second, Nth, List etc.

And there is also last() function which is used when creating a list of items for initializing the array. In the pseudocode it looks like this:

while (some items still remain)
{
  Ptree *e = to take another item;
  Ptree *last = Last(eList);
  last->SetCdr(Cons(e, 0));
}

In general the point is to take the following array’s item and add it to the end of the list. To simplify search of the end we use last() function. OpenC++’s author obviously didn’t suggest that the algorithm will be forced to process the array of millions of items. On small arrays it’s OK, but to the time of such an algorithm increases in proportion to the squared number of items (to be more exact - the triangular number 0.5 * n * (n + 1), where n is the number of items).

The simplest optimization consisting in keeping the pointer to the last item of the list (what allows you not to call last() function) literally worked a miracle. Processing of the file which earlier took more than 3 hours (I didn’t manage to wait more than 3 hours :), now takes several seconds.

Well, all we can say - be careful before you use functions like last(). You can never tell what your algorithm will be sooner or later palmed off. I wish you luck in optimization!

Since SQL Server 2005 we have had the ability to write User Defined Types (UDT's) and User Defined Aggregates (UDA's) etc. with the C# programming language that are executed by the .net framework CLR that is loaded within the SQL Server process.

In SQL Server 2008 the latest service release of the .net framework CLR version 2.0 is loaded within the SQL Server process. Given that the .net framework 4.0 introduces a new version of the CLR, which CLR be loaded by SQL Server if both the 2.0 and 4.0 CLR's are installed?

SQL Server 2008 and the forthcoming SQL Server 2008 R2 release, previously codenamed "Kilimanjaro", will both continue to load the latest service release of the version 2.0 CLR.

You might be wondering, what is the rationale behind SQL Server continuing to load version 2.0 of the CLR? Is it merely a healthy cautious attitude so existing UDT's, UDA's, etc. are not broken; or is it because additional engineering would be required to support the new CLR?

It seems the answer is actually a little of both.

With the .net framework 4.0 we now have the ability to load two or more distinct versions of the CLR within a single process. In previous releases of the .net framework, a process could only load a single instance of the CLR. Given this restriction the CLR team recommended that hosts, such as SQL Server, use the LockClrVersion function to determine the version of the CLR to load prior to initialization. So, as stated previously, SQL Server 2008 and SQL Server 2008 R2 will continue to load the latest service release of the version 2.0 CLR as the version is locked before initialization of the CLR begins.

While future versions of SQL Server may load newer versions of the CLR, or even support the loading of multiple CLR's within the process, version 2.0 of the CLR is here to stay for SQLCLR within SQL Server 2008 and SQL Server 2008 R2.

If you're looking for more information on programming within SQL Server using the .net framework you should read Pro SQL Server 2005 Assemblies by Robbin Dewson and Julian Skinner. Other great books with content related to SQL CLR include Inside Microsoft SQL Server 2008: T-SQL Programming by Itzik Ben-Gan and Programming Microsoft SQL Server 2008 by Leonard Lobel, Andrew J Brust, and Stephen Forte.

Within the SQL Server 2008 Management Studio you can determine the current CLR version by executing the following query against the sys.dm_clr_properties view.

select * from sys.dm_clr_properties

Views are also available within SQL Server 2008 to determine information regarding the application domains, loaded assemblies, and tasks. The results of executing these queries can be seen in Microsoft SQL Server Management Studio below.

select * from sys.dm_clr_appdomains
select * from sys.dm_clr_loaded_assemblies
select * from sys.dm_clr_tasks