2015년 7월 22일 수요일

Data Alignment (copied from )

What is Data Alignment?
In programming language, a data object (variable) has 2 properties; its value and the storage location (address). Data alignment means that the address of a data can be evenly divisible by 1, 2, 4, or 8. In other words, data object can have 1-byte, 2-byte, 4-byte, 8-byte alignment or any power of 2. For instance, if the address of a data is 12FEECh (1244908 in decimal), then it is 4-byte alignment because the address can be evenly divisible by 4. (You can divide it by 2 or 1, but 4 is the highest number that is divisible evenly.)

CPU does not read from or write to memory one byte at a time. Instead, CPU accesses memory in 2, 4, 8, 16, or 32 byte chunks at a time. The reason for doing this is the performance - accessing an address on 4-byte or 16-byte boundary is a lot faster than accessing an address on 1-byte boundary.

The following diagram illustrates how CPU accesses a 4-byte chuck of data with 4-byte memory access granularity.

Memory mapping from memory to CPU cache

If the data is misaligned of 4-byte boundary, CPU has to perform extra work to access the data: load 2 chucks of data, shift out unwanted bytes then combine them together. This process definitely slows down the performance and wastes CPU cycle just to get right data from memory.

Misaligned data slows down data access performance

Structure Member Alignment
In 32-bit x86 systems, the alignment is mostly same as its size of data type. Compiler aligns variables on their natural length boundaries. CPU will handle misaligned data properly, so you do not need to align the address explicitly.

Data alignment for each type
Data Type Alignment(bytes)
char
1
short
2
int
4
float
4
double
4 or 8

However, the story is a little different for member data in struct, union or class objects. The struct (or union, class) member variables must be aligned to the highest bytes of the size of any member variables to prevent performance penalties. For example, if you have 1 char variable (1-byte) and 1 int variable (4-byte) in a struct, the compiler will pads 3 bytes between these two variables. Therefore, the total size of this struct variable is 8 bytes, instead of 5 bytes. By doing this, the address of this struct data is divisible evenly by 4. This is called structure member alignment. Of course, the size of struct will be grown as a consequence.



// size = 2 bytes, alignment = 1-byte, address can be divisible by 1
struct S1 {
char m1; // 1-byte
char m2; // 1-byte
};

// size = 4 bytes, alignment = 2-byte, address can be divisible by 2
struct S2 {
char m1; // 1-byte
// padding 1-byte space here
short m2; // 2-byte
};

// size = 8 bytes, alignment = 4-byte, address can be divisible by 4
struct S3 {
char m1; // 1-byte
// padding 3-byte space here
int m2; // 4-byte
};

// size = 16 bytes, alignment = 8-byte, address can be divisible by 8
struct S4 {
char m1; // 1-byte
// padding 7-byte space here
double m2; // 8-byte
};

// size = 16 bytes, alignment = 8-byte, address can be divisible by 8
struct S5 {
char m1; // 1-byte
// padding 3-byte space here
int m2; // 4-byte
double m2; // 8-byte
};

You may use "pack" pragma directive to specify different packing alignment for struct, union or class members.


Be aware of using custom struct member alignment. It may cause serious compatibility issues, for example, linking external library using different packing alignments. It is better use default alignment all the time.

Data Alignment for SSE
SSE (Streaming SIMD Extensions) defines 128-bit (16-byte) packed data types (4 of 32-bit float data) and access to data can be improved if the address of data is aligned by 16-byte; divisible evenly by 16.

You can declare a variable with 16-byte aligned in MSVC, using __declspec(align(16)) keyword;


__declspec(align(16)) float array[SIZE];
...

struct __declspec(align(16)) S1
{
float v[4];
}

Dynamic array can be allocated using _aligned_malloc() function, and deallocated using _aligned_free().


// allocate 16-byte aligned data
float* array = (float*)_aligned_malloc(SIZE*sizeof(float), 16);
...

// deallocate memory
_aligned_free(array);

Or, you can manually align address like this;


// allocate 15 byte larger array
// because in worst case, the data can be misaligned upto 15 bytes.
float* array = (float*)malloc(SIZE*sizeof(float)+15);

// find the aligned position
// and use this pointer to read or write data into array
float* alignedArray = (float*)(((unsigned long)array + 15) & (~0x0F));
...

// dellocate memory original "array", NOT alignedArray
free(array);
array = alignedArray = 0;

Because 16-byte aligned address must be divisible by 16, the least significant digit in hex number should be 0 all the time. That is why logical operators are used to make the first digit zero in hex number.
Bitwise AND Operator

And, you may have from 0 to 15 bytes misaligned address. In worst case, you have to move the address 15 bytes forward before bitwise AND operation. Therefore, you need to append 15 bytes extra when allocating memory. For example, the 16-byte aligned addresses from 1000h are 1000h, 1010h, 1020h, 1030h, and so on. And if malloc() or C++ new operator allocates a memory space at 1011h, then we need to move 15 bytes forward, which is the next 16-byte aligned address.
Aligned and Misaligned

Copied from http://songho.ca/misc/alignment/dataalign.html.
©2005-2012 Song Ho Ahn (안성호)

2015년 7월 6일 월요일

28-n nas~~

Seoul hotiin kannam duuregiin haa negtee negen coffee shopd orj ireed ongorson doloo honogt bichih baisan temdeglelee nohon bichij suuna. Uchir ni,,,,, 2 jiliin umnuus bi ooriinhoo blogtoo torson odorooroo ongorson neg jiliin hugatsaand uy hiij bvteesnee towchhoon dugnej bichij baihaar tolowloj ehnii 2 jildee yag tuhain uduruu temdeglej bsan bhiim. Harin ene jil yaagaad ch uym we tuhain uduruu bichij amjaagvi unguruuj.Zawgvi law baigaagvi. Za baiz shaltag ni uy bileee hehee. Neg program (Ajiliin bus) ajilluulah gj sonirhoj oroldoj bgaad za ter temdegleliig daraa bichihed boloo shdee gj bodson bhiim. Zaza orshil bvr hachin bolchlooo. Odoo ornol heseg. Ene heseg sonirholtoi mash sonirholtoi bh bh. Yaagaad gewel ongorson neg jilin hugatsaand Jax gedeg zaluugiin 27n nasand mash olon oorchloltvvd shine sodon zvils garsan.


1. Yag neg jiliin umnu tursun uduruuruu erdmiin ajil yaaj bichdeg gedegiig gadarlalaa OU-n paper bichij hewlvvlehiig zorij bn geed bichij bsan. Ter maani 27 nasand mini biellee olsoooon. Jax chadjee hehe. 2014 onii 8 sariin dunduur labiinhaa nztaigaa niilj bichsen erdmiin ajil maani "IEEE cloudcomputing" conference-d accept bolj paper-ee itlgehiin tuld 2014.10.6nd buural europiig zorih ni todorhoi bolowoo.

2. Jax in Europe. Erdem nomiin mor tsagaan gegcheer, hvnd hetsvv sudalgaanii ajil laboratoriin urt udaan ajiliin tsagiin huwaariig tessennii buynd 7 honog erdmiin ajilaa iltgeh nereer europoor 7 honog aylj baigaal irsen dee. Yag vnendee 20 minut l iltgeleee tawiaad vldsen bvh tsagaar ni aylsan. Neg irsniih hehehe.
Aylliin marshrut France -> Luxembourg -> German. Effel toweriig haraad torson ter medremj maani anh Gobi-Altaigaas Ulaanbaatart ireed Sukh janjnii talbaig haraad wooow gj bsantai maash tostei bsan hehe.


3. Togsoh dohlooo, odoo yahuuu ? Za hvn bolgonii tolgoid surguulia togsoh ved iim bodol odor shonogvi ergeldene dee.
Bi ch gesen surguulia togsoh vyeer Gwangju hotoos Seoulruu doloo honog bolgon irj ajiliin yariltslagand orj shalgaltuud ogch blaa. Ur dun ni... Seould ajillahiig hvsdeg bsan gazaraa hvssen chigleliinhee companyd ajild orj chadsan. Magadgvi minii amidraldaa hiisen ololt amjiltiin hamgiin erhmvvdiin neg baih bolow uu. Uchirni minii anhnii ajiliin garaa... Namaig haluun dotnoor hvleen awsan hamt olon maani "NAIM networks" geed startup company.


4.Surguulia togsohoosoo omno ajild orson bolohoor ajiliin talaarhi yariagaa togsoltiinhoo omno oruullaa. 2 sard togsoltiin bayr maani bolj diplomoo awah baisan ch 1 sard togsoltiin ajilaa hamgaalaal compania hurdan ir gj shaardsanii ulmaas, gertee neg harichihaad irii gehed daraa boloo geeed yawuulaagvin ulmaas hehe, shuud ajildaa orson. Ajillaad barag 2 sar bolsnii daraa neg odoriin choloo awch togsoltiin bayrtaa orson mini ene.


5. Solongos ulsiin Gwangju hotod 7n jil shaham amidraad Seould irj amidarlaaa. Nvvdel, shine orchin, shine hvmvvs, hedii bainga seould irj ochij bdag bsan ch olon jil amidarsan gazartaaa dendvv dassan bsan. Seould yalanguyaa ajildaa oir geed kannamd tvr amidrahaar bolson nadad endhiin amidral nileed vnetei sanagdsan. Udahgvi nvvne dee. Gehdeee zaluu hvnd iim gazar tvr ch gesen amidarch vzehni goy ch uym shig. Vnetei mashin, ondor barilga, gangan hvmvvsiig harahaad bi ch gesen ingej amidarna daa gesen neg tiim inspiration bainga medregdene hehe.


Ene jil nileen urt bichijeee. Minii urd bgaa 28-n nas mini ongorson jilees ilvv ololt amjiltiig nadad awchrah boltugai gj helelgvi bailgahiin toloo hicheenee, ooriin chadah bvhnee daichilnaa gj heleed onoodriin temdeglelee ondorlii.

Aan tiin neeree, minii torson odor ene jil 3 dahi odor bolson. Jil bolgon neg odoroor uragshlaad bdgiin bnlee. Za tegeed bas ene jilees caketai awhuulsan zurgaa bas hiij baihaar shiidsen.

2015.07.06 Seoul, South Korea

2015년 7월 1일 수요일

Installing and Using Distributed ONOS

1. Update
$ sudo apt-get update
2. Installing git
$ sudo apt-get install git
$ sudo git init
3. Downloading ONOS
$ git clone https://gerrit.onosproject.org/onos
4. Change the branch
$ cd onos
$ git checkout onos-1.1 # checks out the Blackbird maintenance branch
# !!! If you have already cloned an earlier version of the repository, you can update it using following git commands
$ cd onos
$ git checkout master # checks out 'master' as the working branch
$ git pull --rebase origin master # pulls the latest code from Gerrit, and rebases any local changes onto origin/master
5. Adding dependency
$ sudo apt-get install vim
$ sudo vim pom.xml
# add following inside the dependencies part

org.onosproject
onos-api
1.1.1-SNAPSHOT

6. Installing – Maven, Karaf
$ cd
$ sudo mkdir Downloads Applications
$ cd Downloads
$sudo wget http://apache.mirror.cdnetworks.com/karaf/3.0.2/apache-karaf-3.0.2.tar.gz
$sudo wget http://mirror.apache-kr.org/maven/maven-3/3.3.1/binaries/apache-maven-3.3.1-bin.tar.gz
$sudo tar -zxvf apache-karaf-3.0.2.tar.gz -C ~/Applications/
$sudo tar -zxvf apache-maven-3.3.1-bin.tar.gz -C ~/Applications/

7. Installing Java
$ sudo apt-get install software-properties-common -y
$ sudo add-apt-repository ppa:webupd8team/java -y
$ sudo apt-get update
$ sudo apt-get install oracle-java8-installer oracle-java8-set-default –y
#Accept Oracle license agreement
8. Setting JAVA_HOME
$ export JAVA_HOME=/usr/lib/jvm/java-8-oracle
$ env | grep JAVA_HOME
JAVA_HOME=/usr/lib/jvm/java-8-oracle
9. Installing Maven
$sudo apt-get install maven -y
10. Setting no password for the user (recommendation: name user as sdn)
$ su – sdn # sdn is user name
$ password: #enter password
$ ssh-keygen –t rsa #do not put any password, just press on enter till complete
$ ssh-copy-id sdn@server IP address
$ ssh sdn@server IP address #checking
$ sudo vim /etc/sudoers

#Input following:
sdn ALL=(ALL) NOPASSWD:ALL
#save (wq!)
11. Installing ONOS
$ export ONOS_ROOT=~/onos
$ source $ONOS_ROOT/tools/dev/bash_profile
$ cd ~/onos
$ mvn clean install # No sudo!!! With sudo, .m2/repository is not generated

Turn off the VM, and copy the VM to create other controllers (create other VMs using generated VM.)

12. Start Distributed ONOS
$ export ONOS_ROOT=~/onos
$ source $ONOS_ROOT/tools/dev/bash_profile
$ export ONOS_USER=sdn
$ export ONOS_APPS=“drivers,openflow,fwd,proxyarp,mobility”
$ export ONOS_NIC=192.168.25.*
$onos-package
$onos-patch-vm 192.168.25.150
$onos-patch-vm 192.168.25.151
$onos-install -f 192.168.25.150
$onos-install -f 192.168.25.151
Once up, we needed to modify a few files on each server and restart the servers. By default ONOS uses 192.168.56.* as the IP address blocks, so we had to modify them to match our setup. My setup was like following.
/opt/onos/config/cluster.json

cluster.json
{ "ipPrefix": "192.168.25.*",
"nodes":[
{ "id": "192.168.25.150", "ip": "192.168.25.150", "tcpPort": 9876 },
{ "id": "192.168.25.151", "ip": "192.168.21.151", "tcpPort": 9876 }
]}
and /opt/onos/config/tablets.json
{
"nodes": [
{
"ip": "192.168.25.150",
"id": "192.168.25.150",
"tcpPort": 7238
},
{
"ip": "192.168.25.151",
"id": "192.168.25.151",
"tcpPort": 7238
}
],
"partitions": {
"p1": [
{
"ip": "192.168.25.150",
"id": "192.168.25.150",
"tcpPort": 7238
},
{
"ip": "192.168.25.151",
"id": "192.168.21.151",
"tcpPort": 7238
}
],
"p2": [
{
"ip": "192.168.25.151",
"id": "192.168.25.151",
"tcpPort": 7238
},
{
"ip": "192.168.25.150",
"id": "192.168.25.150",
"tcpPort": 7238
}
]
}
}