2015년 9월 23일 수요일

THE PING PROCESS

  1. The source host generates an ICMP protocol data unit.
  2. The ICMP PDU is encapsulated in an IP datagram, with the source and destination IP addresses in the IP header. At this point the datagram is most properly referred to as an ICMP ECHO datagram, but we will call it an IP datagram from here on since that's what it looks like to the networks it is sent over.
  3. The source host notes the local time on it's clock as it transmits the IP datagram towards the destination. Each host that receives the IP datagram checks the destination address to see if it matches their own address or is the all hosts address (all 1's in the host field of the IP address).
  4. If the destination IP address in the IP datagram does not match the local host's address, the IP datagram is forwarded to the network where the IP address resides.
  5. The destination host receives the IP datagram, finds a match between itself and the destination address in the IP datagram.
  6. The destination host notes the ICMP ECHO information in the IP datagram, performs any necessary work then destroys the original IP/ICMP ECHO datagram.
  7. The destination host creates an ICMP ECHO REPLY, encapsulates it in an IP datagram placing it's own IP address in the source IP address field, and the original sender's IP address in the destination field of the IP datagram.
  8. The new IP datagram is routed back to the originator of the PING. The host receives it, notes the time on the clock and finally prints PING output information, including the elapsed time

What are the series of steps that happen when an URL is requested from the address field of a browser?

This is a question whose answer could grow into an entire course on networking, so here's a version that only details some of the cases. There could probably be followup questions.


  1. The browser extracts the domain name from the URL.
  2. The browser queries DNS for the IP address of the URL. Generally, the browser will have cached domains previously visited, and the operating system will have cached queries from any number of applications. If neither the browser nor the OS have a cached copy of the IP address, then a request is sent off to the system's configured DNS server. The client machine knows the IP address for the DNS server, so no lookup is necessary.
  3. The request sent to the DNS server is almost always smaller than the maximum packet size, and is thus sent off as a single packet. In addition to the content of the request, the packet includes the IP address it is destined for in its header. Except in the simplest of cases (network hubs), as the packet reaches each piece of network equipment between the client and server, that equipment uses a routing table to figure out what node it is connected to that is most likely to be part of the fastest route to the destination. The process of determining which path is the best choice differs between equipment and can be very complicated.
  4. The is either lost (in which case the request fails or is reiterated), or makes it to its destination, the DNS server.
  5. If that DNS server has the address for that domain, it will return it. Otherwise, it will forward the query along to DNS server it is configured to defer to. This happens recursively until the request is fulfilled or it reaches an authoritative name server and can go no further. (If the authoritative name server doesn't recognize the domain, the response indicates failure and the browser generally gives an error like "Can't find the server atwww.lkliejafadh.com".) The response makes its way back to the client machine much like the request traveled to the DNS server.
  6. Assuming the DNS request is successful, the client machine now has an IP address that uniquely identifies a machine on the Internet. The web browser then assembles an HTTP request, which consists of a header and optional content. The header includes things like the specific path being requested from the web server, the HTTP version, any relevant browser cookies, etc. In the case in question (hitting Enter in the address bar), the content will be empty. In other cases, it may include form data like a username and password (or the content of an image file being uploaded,  
  7. This HTTP request is sent off to the web server host as some number of packets, each of which is routed in the same was as the earlier DNS query. (The packets have sequence numbers that allow them to be reassembled in order even if they take different paths.) Once the request arrives at the webserver, it generates a response (this may be a static page, served as-is, or a more dynamic response, generated in any number of ways.) The web server software sends the generated page back to the client.
  8. Assuming the response HTML and not an image or data file, then the browser parses the HTML to render the page. Part of this parsing and rendering process may be the discovery that the web page includes images or other embedded content that is not part of the HTML document. The browser will then send off further requests (either to the original web server or different ones, as appropriate) to fetch the embedded content, which will then be rendered into the document as well.
Copied from https://www.quora.com/What-are-the-series-of-steps-that-happen-when-an-URL-is-requested-from-the-address-field-of-a-browser

2015년 8월 22일 토요일

2015년 7월 22일 수요일

Data Alignment (copied from )

What is Data Alignment?
In programming language, a data object (variable) has 2 properties; its value and the storage location (address). Data alignment means that the address of a data can be evenly divisible by 1, 2, 4, or 8. In other words, data object can have 1-byte, 2-byte, 4-byte, 8-byte alignment or any power of 2. For instance, if the address of a data is 12FEECh (1244908 in decimal), then it is 4-byte alignment because the address can be evenly divisible by 4. (You can divide it by 2 or 1, but 4 is the highest number that is divisible evenly.)

CPU does not read from or write to memory one byte at a time. Instead, CPU accesses memory in 2, 4, 8, 16, or 32 byte chunks at a time. The reason for doing this is the performance - accessing an address on 4-byte or 16-byte boundary is a lot faster than accessing an address on 1-byte boundary.

The following diagram illustrates how CPU accesses a 4-byte chuck of data with 4-byte memory access granularity.

Memory mapping from memory to CPU cache

If the data is misaligned of 4-byte boundary, CPU has to perform extra work to access the data: load 2 chucks of data, shift out unwanted bytes then combine them together. This process definitely slows down the performance and wastes CPU cycle just to get right data from memory.

Misaligned data slows down data access performance

Structure Member Alignment
In 32-bit x86 systems, the alignment is mostly same as its size of data type. Compiler aligns variables on their natural length boundaries. CPU will handle misaligned data properly, so you do not need to align the address explicitly.

Data alignment for each type
Data Type Alignment(bytes)
char
1
short
2
int
4
float
4
double
4 or 8

However, the story is a little different for member data in struct, union or class objects. The struct (or union, class) member variables must be aligned to the highest bytes of the size of any member variables to prevent performance penalties. For example, if you have 1 char variable (1-byte) and 1 int variable (4-byte) in a struct, the compiler will pads 3 bytes between these two variables. Therefore, the total size of this struct variable is 8 bytes, instead of 5 bytes. By doing this, the address of this struct data is divisible evenly by 4. This is called structure member alignment. Of course, the size of struct will be grown as a consequence.



// size = 2 bytes, alignment = 1-byte, address can be divisible by 1
struct S1 {
char m1; // 1-byte
char m2; // 1-byte
};

// size = 4 bytes, alignment = 2-byte, address can be divisible by 2
struct S2 {
char m1; // 1-byte
// padding 1-byte space here
short m2; // 2-byte
};

// size = 8 bytes, alignment = 4-byte, address can be divisible by 4
struct S3 {
char m1; // 1-byte
// padding 3-byte space here
int m2; // 4-byte
};

// size = 16 bytes, alignment = 8-byte, address can be divisible by 8
struct S4 {
char m1; // 1-byte
// padding 7-byte space here
double m2; // 8-byte
};

// size = 16 bytes, alignment = 8-byte, address can be divisible by 8
struct S5 {
char m1; // 1-byte
// padding 3-byte space here
int m2; // 4-byte
double m2; // 8-byte
};

You may use "pack" pragma directive to specify different packing alignment for struct, union or class members.


Be aware of using custom struct member alignment. It may cause serious compatibility issues, for example, linking external library using different packing alignments. It is better use default alignment all the time.

Data Alignment for SSE
SSE (Streaming SIMD Extensions) defines 128-bit (16-byte) packed data types (4 of 32-bit float data) and access to data can be improved if the address of data is aligned by 16-byte; divisible evenly by 16.

You can declare a variable with 16-byte aligned in MSVC, using __declspec(align(16)) keyword;


__declspec(align(16)) float array[SIZE];
...

struct __declspec(align(16)) S1
{
float v[4];
}

Dynamic array can be allocated using _aligned_malloc() function, and deallocated using _aligned_free().


// allocate 16-byte aligned data
float* array = (float*)_aligned_malloc(SIZE*sizeof(float), 16);
...

// deallocate memory
_aligned_free(array);

Or, you can manually align address like this;


// allocate 15 byte larger array
// because in worst case, the data can be misaligned upto 15 bytes.
float* array = (float*)malloc(SIZE*sizeof(float)+15);

// find the aligned position
// and use this pointer to read or write data into array
float* alignedArray = (float*)(((unsigned long)array + 15) & (~0x0F));
...

// dellocate memory original "array", NOT alignedArray
free(array);
array = alignedArray = 0;

Because 16-byte aligned address must be divisible by 16, the least significant digit in hex number should be 0 all the time. That is why logical operators are used to make the first digit zero in hex number.
Bitwise AND Operator

And, you may have from 0 to 15 bytes misaligned address. In worst case, you have to move the address 15 bytes forward before bitwise AND operation. Therefore, you need to append 15 bytes extra when allocating memory. For example, the 16-byte aligned addresses from 1000h are 1000h, 1010h, 1020h, 1030h, and so on. And if malloc() or C++ new operator allocates a memory space at 1011h, then we need to move 15 bytes forward, which is the next 16-byte aligned address.
Aligned and Misaligned

Copied from http://songho.ca/misc/alignment/dataalign.html.
©2005-2012 Song Ho Ahn (안성호)

2015년 7월 6일 월요일

28-n nas~~

Seoul hotiin kannam duuregiin haa negtee negen coffee shopd orj ireed ongorson doloo honogt bichih baisan temdeglelee nohon bichij suuna. Uchir ni,,,,, 2 jiliin umnuus bi ooriinhoo blogtoo torson odorooroo ongorson neg jiliin hugatsaand uy hiij bvteesnee towchhoon dugnej bichij baihaar tolowloj ehnii 2 jildee yag tuhain uduruu temdeglej bsan bhiim. Harin ene jil yaagaad ch uym we tuhain uduruu bichij amjaagvi unguruuj.Zawgvi law baigaagvi. Za baiz shaltag ni uy bileee hehee. Neg program (Ajiliin bus) ajilluulah gj sonirhoj oroldoj bgaad za ter temdegleliig daraa bichihed boloo shdee gj bodson bhiim. Zaza orshil bvr hachin bolchlooo. Odoo ornol heseg. Ene heseg sonirholtoi mash sonirholtoi bh bh. Yaagaad gewel ongorson neg jilin hugatsaand Jax gedeg zaluugiin 27n nasand mash olon oorchloltvvd shine sodon zvils garsan.


1. Yag neg jiliin umnu tursun uduruuruu erdmiin ajil yaaj bichdeg gedegiig gadarlalaa OU-n paper bichij hewlvvlehiig zorij bn geed bichij bsan. Ter maani 27 nasand mini biellee olsoooon. Jax chadjee hehe. 2014 onii 8 sariin dunduur labiinhaa nztaigaa niilj bichsen erdmiin ajil maani "IEEE cloudcomputing" conference-d accept bolj paper-ee itlgehiin tuld 2014.10.6nd buural europiig zorih ni todorhoi bolowoo.

2. Jax in Europe. Erdem nomiin mor tsagaan gegcheer, hvnd hetsvv sudalgaanii ajil laboratoriin urt udaan ajiliin tsagiin huwaariig tessennii buynd 7 honog erdmiin ajilaa iltgeh nereer europoor 7 honog aylj baigaal irsen dee. Yag vnendee 20 minut l iltgeleee tawiaad vldsen bvh tsagaar ni aylsan. Neg irsniih hehehe.
Aylliin marshrut France -> Luxembourg -> German. Effel toweriig haraad torson ter medremj maani anh Gobi-Altaigaas Ulaanbaatart ireed Sukh janjnii talbaig haraad wooow gj bsantai maash tostei bsan hehe.


3. Togsoh dohlooo, odoo yahuuu ? Za hvn bolgonii tolgoid surguulia togsoh ved iim bodol odor shonogvi ergeldene dee.
Bi ch gesen surguulia togsoh vyeer Gwangju hotoos Seoulruu doloo honog bolgon irj ajiliin yariltslagand orj shalgaltuud ogch blaa. Ur dun ni... Seould ajillahiig hvsdeg bsan gazaraa hvssen chigleliinhee companyd ajild orj chadsan. Magadgvi minii amidraldaa hiisen ololt amjiltiin hamgiin erhmvvdiin neg baih bolow uu. Uchirni minii anhnii ajiliin garaa... Namaig haluun dotnoor hvleen awsan hamt olon maani "NAIM networks" geed startup company.


4.Surguulia togsohoosoo omno ajild orson bolohoor ajiliin talaarhi yariagaa togsoltiinhoo omno oruullaa. 2 sard togsoltiin bayr maani bolj diplomoo awah baisan ch 1 sard togsoltiin ajilaa hamgaalaal compania hurdan ir gj shaardsanii ulmaas, gertee neg harichihaad irii gehed daraa boloo geeed yawuulaagvin ulmaas hehe, shuud ajildaa orson. Ajillaad barag 2 sar bolsnii daraa neg odoriin choloo awch togsoltiin bayrtaa orson mini ene.


5. Solongos ulsiin Gwangju hotod 7n jil shaham amidraad Seould irj amidarlaaa. Nvvdel, shine orchin, shine hvmvvs, hedii bainga seould irj ochij bdag bsan ch olon jil amidarsan gazartaaa dendvv dassan bsan. Seould yalanguyaa ajildaa oir geed kannamd tvr amidrahaar bolson nadad endhiin amidral nileed vnetei sanagdsan. Udahgvi nvvne dee. Gehdeee zaluu hvnd iim gazar tvr ch gesen amidarch vzehni goy ch uym shig. Vnetei mashin, ondor barilga, gangan hvmvvsiig harahaad bi ch gesen ingej amidarna daa gesen neg tiim inspiration bainga medregdene hehe.


Ene jil nileen urt bichijeee. Minii urd bgaa 28-n nas mini ongorson jilees ilvv ololt amjiltiig nadad awchrah boltugai gj helelgvi bailgahiin toloo hicheenee, ooriin chadah bvhnee daichilnaa gj heleed onoodriin temdeglelee ondorlii.

Aan tiin neeree, minii torson odor ene jil 3 dahi odor bolson. Jil bolgon neg odoroor uragshlaad bdgiin bnlee. Za tegeed bas ene jilees caketai awhuulsan zurgaa bas hiij baihaar shiidsen.

2015.07.06 Seoul, South Korea

2015년 7월 1일 수요일

Installing and Using Distributed ONOS

1. Update
$ sudo apt-get update
2. Installing git
$ sudo apt-get install git
$ sudo git init
3. Downloading ONOS
$ git clone https://gerrit.onosproject.org/onos
4. Change the branch
$ cd onos
$ git checkout onos-1.1 # checks out the Blackbird maintenance branch
# !!! If you have already cloned an earlier version of the repository, you can update it using following git commands
$ cd onos
$ git checkout master # checks out 'master' as the working branch
$ git pull --rebase origin master # pulls the latest code from Gerrit, and rebases any local changes onto origin/master
5. Adding dependency
$ sudo apt-get install vim
$ sudo vim pom.xml
# add following inside the dependencies part

org.onosproject
onos-api
1.1.1-SNAPSHOT

6. Installing – Maven, Karaf
$ cd
$ sudo mkdir Downloads Applications
$ cd Downloads
$sudo wget http://apache.mirror.cdnetworks.com/karaf/3.0.2/apache-karaf-3.0.2.tar.gz
$sudo wget http://mirror.apache-kr.org/maven/maven-3/3.3.1/binaries/apache-maven-3.3.1-bin.tar.gz
$sudo tar -zxvf apache-karaf-3.0.2.tar.gz -C ~/Applications/
$sudo tar -zxvf apache-maven-3.3.1-bin.tar.gz -C ~/Applications/

7. Installing Java
$ sudo apt-get install software-properties-common -y
$ sudo add-apt-repository ppa:webupd8team/java -y
$ sudo apt-get update
$ sudo apt-get install oracle-java8-installer oracle-java8-set-default –y
#Accept Oracle license agreement
8. Setting JAVA_HOME
$ export JAVA_HOME=/usr/lib/jvm/java-8-oracle
$ env | grep JAVA_HOME
JAVA_HOME=/usr/lib/jvm/java-8-oracle
9. Installing Maven
$sudo apt-get install maven -y
10. Setting no password for the user (recommendation: name user as sdn)
$ su – sdn # sdn is user name
$ password: #enter password
$ ssh-keygen –t rsa #do not put any password, just press on enter till complete
$ ssh-copy-id sdn@server IP address
$ ssh sdn@server IP address #checking
$ sudo vim /etc/sudoers

#Input following:
sdn ALL=(ALL) NOPASSWD:ALL
#save (wq!)
11. Installing ONOS
$ export ONOS_ROOT=~/onos
$ source $ONOS_ROOT/tools/dev/bash_profile
$ cd ~/onos
$ mvn clean install # No sudo!!! With sudo, .m2/repository is not generated

Turn off the VM, and copy the VM to create other controllers (create other VMs using generated VM.)

12. Start Distributed ONOS
$ export ONOS_ROOT=~/onos
$ source $ONOS_ROOT/tools/dev/bash_profile
$ export ONOS_USER=sdn
$ export ONOS_APPS=“drivers,openflow,fwd,proxyarp,mobility”
$ export ONOS_NIC=192.168.25.*
$onos-package
$onos-patch-vm 192.168.25.150
$onos-patch-vm 192.168.25.151
$onos-install -f 192.168.25.150
$onos-install -f 192.168.25.151
Once up, we needed to modify a few files on each server and restart the servers. By default ONOS uses 192.168.56.* as the IP address blocks, so we had to modify them to match our setup. My setup was like following.
/opt/onos/config/cluster.json

cluster.json
{ "ipPrefix": "192.168.25.*",
"nodes":[
{ "id": "192.168.25.150", "ip": "192.168.25.150", "tcpPort": 9876 },
{ "id": "192.168.25.151", "ip": "192.168.21.151", "tcpPort": 9876 }
]}
and /opt/onos/config/tablets.json
{
"nodes": [
{
"ip": "192.168.25.150",
"id": "192.168.25.150",
"tcpPort": 7238
},
{
"ip": "192.168.25.151",
"id": "192.168.25.151",
"tcpPort": 7238
}
],
"partitions": {
"p1": [
{
"ip": "192.168.25.150",
"id": "192.168.25.150",
"tcpPort": 7238
},
{
"ip": "192.168.25.151",
"id": "192.168.21.151",
"tcpPort": 7238
}
],
"p2": [
{
"ip": "192.168.25.151",
"id": "192.168.25.151",
"tcpPort": 7238
},
{
"ip": "192.168.25.150",
"id": "192.168.25.150",
"tcpPort": 7238
}
]
}
}

2014년 8월 21일 목요일

Installing Prelude (HIDS) with Suricata NIDS from packages on Ubuntu-14.04.1-desktop-amd64

Prelude is a Universal "Security Information Event Management" (SIEM) system. It collects, archives, normalizes, sorts, aggregates, correlates and reports all security-related events independently of the product brand or license giving rise to such events.  Prelude will allow to log all of the events to the prelude database and be consulted using one interface (prewikka).  Shortly, we can monitor a  number of IDSs on the one central interface. 

Prelude simple architecture (Sensor A, B, C could be Snort, Suricata, OSSEC, SanCP)



At first time, I had some mysql version and root user related errors, but it is solved by removing previous mysql-servert. #sudo apt-get purge mysql-server (If you installed mysql before on you pc). One more thing is do not upgrade the ubuntu, bcz of this version related error!!!

Followings are simple steps to setup Prelude HIDS
  1. # sudo su (change user to root)
  2. # apt-get update
  3. # apt-get install ntpdate
  4. # apt-get install dbconfig-common
  5. # apt-get install rng-tools (Edit vi /etc/default/rng-tools ->HRNGDEVICE=/dev/urandom)
  6. # apt-get install mysql-server (remember root password!!!)
  7. # apt-get install prelude-manager    
  8. # vi /etc/default/prelude-manager ->RUN= yes
  9. # vi /etc/prelude-manager/prelude-manager.conf  (change the default listen address into th sensor IP)   
  10. # vi  /etc/prelude/default/client.conf (change the server IP into your IP)
  11. # /etc/init.d/prelude-manager stop
  12. # /etc/init.d/prelude-manager start
  13. # ps auxw | grep manager                                                                                                                                                                prelude 13178 0.0 0.2 213596 2708 ? Ssl 15:01 0:02 /usr/sbin/prelude-manager -d -P /var/run/prelude-manager.pid
  14. # netstat -pantu | grep prelude                                                                                                                                                                tcp  0   0   192.168.10.155:4690   0.0.0.0:*    LISTEN      13178/prelude-manag
  15. Need to install prelude-lml on every host you want to monitor. Prelude-LML will analyze your logs and reports event to the managers. # apt-get install prelude-lml                                                                                                                                                   On the LML client, run the register command:                                                                                                                                  #prelude-addmin register prelude-lml "idmef:w" <manager address> --uid 0 --gid 0                                                                    On the manager, run the following:                                                                                                                                                        #prelude-adduser registration-server prelude-manager                                                               
  16. # /etc/init.d/prelude-lml start                                                                                                                                                                 Starting Prelude LML: prelude-lml                                                        
  17. # ps auxw | grep lml                                                                                                                                                                             root 1946 0.3 0.0 20856 3424 ? Ss 14:35 0:00 /usr/bin/prelude-lml -d -q -P /var/run/prelude-lml.pid
  18. # apt-get install prewikka
  19. Get the password from prelude-manager configuration file /etc/prelude manager/prelude-manager.conf and edit prewikka configuration file /etc/prewikka/prewikka.conf:   #vi /etc/prewikka/prewikka.conf (change idmef_database pass!!!)
  20. # apt-get install libapache2-mod-python
  21. # apt-get install apache2
  22. # vi /etc/apache2/sites-available/prewikka  (put following)                                                                                                                  <VirtualHost *:80>
     Setenv PREWIKKA_CONFIG "/etc/prewikka/prewikka.conf"
    <Location "/">
            AllowOverride None
            Options ExecCGI
            <IfModule mod_mime.c>
                    AddHandler cgi-script .cgi
            </IfModule>
            Order allow,deny
            Allow from all
    </Location>
    Alias /prewikka/ /usr/share/prewikka/htdocs/ ScriptAlias / /usr/share/prewikka/cgi-bin/prewikka.cgi
    </VirtualHost>                                                                               
  23. # chmod 766 /etc/prewikka/prewikka.conf
  24. # a2dissite
  25. # a2ensite 000-default default-ssl
  26. # /etc/init.d/apache2 reload
  27. # prewikka-httpd
  28. On the browser localhost:8000 or "manager ip":8000
                                               

3rd party agent installation (Installing Suricata and configuring it for Prelude)
  1. #apt-get install suricata
  2. Edit /etc/suricata/suricata-debian.yaml file to enable prelude alert :                                                                                                 #vi /etc/suricata/suricata-debian.yaml     
      - alert-prelude:
          enabled: yes
          profile: suricata
          log-packet-content: yes
          log-packet-header: yes
  3. Registering Suricata profile                                                                                                                                                                    On the suricata host :                                                                                                                                                                           # prelude-admin register suricata "idmef:w" <manager IP address> --uid 0 --gid 0                                                                            On the Prelude-Manager host :                                                                                                                                                      # prelude-admin registration-server prelude-manager                                                                                                                    After your RSA key creation, a password is generated. Paste it from the second terminal to the first terminal, then confirm the registration on the second terminal                                            
  4. Running Suricata                                                                                                                                                                                  # LD_LIBRARY_PATH=/usr/lib /usr/bin/suricata -c /etc/suricata//suricata-debian.yaml -i eth0
  5. Do step 25-28 again