engin aydogan

Making multiple VPN connections and using them at the same time

engin — Fri, 09 Mar 2012 20:56:52 +0000

As I figured how to make certain clients on LAN to use a VPN, I was a bit unsatisfied with the performance of the VPN connection (read: low transfer rates). Then I remembered that we can add multiple routes (explained here). Which means one connection a client makes are routed via VPN0, the other one is routed from VPN1. Hell, you can even add same VPN again and again so that it is used more often.

So as an addition to my previous post, you can simple add more VPN connections in interfaces. Then at the same routing rule, you can add all those VPN connection as gateways. It looks like this:

[admin@MikroTik] > ip route print
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 A S  0.0.0.0/0                          pptp-out2          1
                                           pptp-out1
                                           pptp-out3

It is the rule that matches the packets with route-mark, I added more gateways to it. Now when a connection is made, it is routed via one of these.

Now, you can use all the cumulative bandwidth all these servers allocated for you.

Though, I couldn’t make burst of multiple connection attempts distributed over these gateways yet. Which means, if a client opens many connections rapidly, they are all routed from one gateway. If I can figure this out, a segmented downloading for instance, would be MUCH faster with many VPN servers utilized.

Making certain clients on LAN use a particular VPN connection with Mikrotik

engin — Fri, 09 Mar 2012 19:15:35 +0000

Today, I received my RB-750GL from (kablosuzmarket.com) within 24h of my purchase, kudos for the excellent service. Rushed home to use this beast.

I had this idea for some time. To make certain devices on the LAN (i.e. Apple TV) to use a VPN connection so they appear to be from US (or whever the VPN server is). With Mikrotik it took just about half an hour to figure out how to do this.

1. Add your VPN connection into interface. In my case, I needed PPTP Client. Make sure it’s connected successfully.

[admin@MikroTik] /interface> print
Flags: D - dynamic, X - disabled, R - running, S - slave
 #     NAME                                          TYPE             MTU   L2MTU
 0  R  ether1-gateway                                ether            1500  1598
 1  R  ether2-local-master                           ether            1500  1598
 2  R  ether3-local-slave                            ether            1500  1598
 3     ether4-local-slave                            ether            1500  1598
 4     ether5-local-slave                            ether            1500  1598
 5  R  pppoe-out1                                    pppoe-out        1480
 6  R  pptp-out1                                     pptp-out         1404

Last one is my VPN connection.
2. Prepare the address list of the devices you want to use this new VPN connection.

[admin@MikroTik] /ip firewall address-list> print
Flags: X - disabled, D - dynamic
 #   LIST                                         ADDRESS
 0   usvpn-addrlist                               192.168.1.104
 1   usvpn-addrlist                               192.168.1.254

3. Add firewall rules to mark-route packets from this address list. I’ve marked them as “usvpn”.

[admin@MikroTik] /ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic
 0   chain=prerouting action=mark-routing new-routing-mark=usvpn passthrough=yes src-address-list=usvpn-addrlist

4. Now, route packets marked with “usvpn” route-mark via pptp1 interface.

[admin@MikroTik] > ip route print
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 A S  0.0.0.0/0                          pptp-out1          1
 1 ADS  0.0.0.0/0                          78.171.192.1       1
 2 ADC  10.10.0.1/32       10.10.0.2       pptp-out1          0
 3 ADC  10.10.2.1/32       10.10.43.56     pppoe-out1         0
 4 ADC  192.168.1.0/24     192.168.1.1     ether2-local-ma... 0

The details are not visible in this print. When adding new route, just select the Gateway (pptp1) and Routing Mark (usvpn).

Now test if your devices are out in the US :)

Windows, PHP, cURL SSL certificate problem.

engin — Thu, 02 Feb 2012 21:22:31 +0000

When you use a library that wraps an API (Facebook PHP SDK, AWS PHP SDK) usage, it often boils down to cURL in the core. These libraries often want to communicate over SSL by default. So, any call you make on these libraries involves a cURL request over a SSL connection. The problem with this is that cURL does not ship CA certificates bundle any more, so SSL certificates of the sites the requests are addressed to cannot be verified.

For instance, with AWS PHP SDK, you’ll get the following error if your cURL setup is not fixed:

Fatal error: Uncaught exception 'cURL_Exception' with message 'cURL resource: Resource id #10; cURL error: SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed (cURL error code 60). See http://curl.haxx.se/libcurl/c/libcurl-errors.html for an explanation of error codes.' in C:\Users\Engin\Code\PHP\unuttumbile\AWSSDKforPHP\lib\requestcore\requestcore.class.php:824
Stack trace:
#0 C:\Users\Engin\Code\PHP\unuttumbile\AWSSDKforPHP\services\s3.class.php(895): RequestCore->send_request()
#1 C:\Users\Engin\Code\PHP\unuttumbile\AWSSDKforPHP\services\s3.class.php(1125): AmazonS3->authenticate('php-sdk-getting...', Array)
#2 C:\Users\Engin\Code\PHP\unuttumbile\AWSSDKforPHP\_samples\cli-s3_get_urls_for_uploads.php(60): AmazonS3->create_bucket('php-sdk-getting...', 's3-us-west-1.am...')
#3 {main}
thrown in C:\Users\Engin\Code\PHP\unuttumbile\AWSSDKforPHP\lib\requestcore\requestcore.class.php on line 824

cURL explains the situation in depth here.

2 step easy solution:

Generate CA bundle yourself via scripts provided by cURL. Either via mk-ca-bundle.pl (PERL) or mk-ca-bundle.vbs (Windows Shell). Then copy it to c:\Windows just for convenience.

Engin@Engin-VAIO ~
$ perl mk-ca-bundle.pl
Downloading 'certdata.txt' ...
Processing  'certdata.txt' ...
Done (137 CA certs processed, 32 untrusted skipped).

Engin@Engin-VAIO ~
$ cp ca-bundle.crt /cygdrive/c/Windows/ -vf
`ca-bundle.crt' -> `/cygdrive/c/Windows/ca-bundle.crt'

Then add the following line in your php.ini’s [PHP] section. Note that this configuration option is available since PHP 5.3.7. See not so very detailed info.
```
[PHP]
curl.cainfo = c:\windows\ca-bundle.crt
```
Restart your HTTP server and you’re done.

9Gagtension – Facebook comment, message and chat trendy meme emoticons support

engin — Fri, 16 Dec 2011 06:39:44 +0000

I noticed I was using quite some meme references in my Facebook comments such as :sweet jesus: or :mother of god:. So, yesterday evening, I put together an Chrome extension for this. Very simple.

List of memes and other information: http://engin.bzzzt.biz/9gagtension

Source code: https://github.com/engina/9gagtension

This is where you install: Chrome Extension Page

Windows routing problem: Network is unreachable

engin — Tue, 13 Dec 2011 08:57:51 +0000

I don’t know what got broken, or how did it get broken but IP connections on my interface which is using AUTO IP (169.254.0.0/16) stopped working all of a sudden.

$ nc  -nvvvv 169.254.2.206 23
(UNKNOWN) [169.254.2.206] 23 (?) : Network is unreachable
 sent 0, rcvd 0

When I explicitly bind to the correct interface it worked.

$ nc -s 169.254.46.68 -nvvvv 169.254.2.206 23
(UNKNOWN) [169.254.2.206] 23 (?) open
Welcome to ENDA Administration Terminal
Password:

The fact that the error is returned immediately and the content of the error points out that it is a routing problem. So I checked my routing table.

$ route -4 print
===========================================================================
Interface List
 37...00 1f 1f ea 7f 52 ......150Mbps Wireless 802.11b/g/n Nano USB Adapter
 28...00 ff 2d 67 65 d4 ......TAP-Win32 Adapter V9
 10...00 1d ba 68 a5 ac ......Intel(R) 82567LM Gigabit Network Connection
  1...........................Software Loopback Interface 1
 12...00 00 00 00 00 00 00 e0 Teredo Tunneling Pseudo-Interface
 39...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter
 26...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #3
 38...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #2
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0      192.168.1.1    192.168.1.104     25
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    306
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    306
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    306
    169.254.46.68  255.255.255.255         On-link     169.254.46.68    356
      192.168.1.0    255.255.255.0         On-link     192.168.1.104    281
    192.168.1.104  255.255.255.255         On-link     192.168.1.104    281
    192.168.1.255  255.255.255.255         On-link     192.168.1.104    281
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    306
        224.0.0.0        240.0.0.0         On-link     169.254.46.68    356
        224.0.0.0        240.0.0.0         On-link     192.168.1.104    281
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    306
  255.255.255.255  255.255.255.255         On-link     169.254.46.68    356
  255.255.255.255  255.255.255.255         On-link     192.168.1.104    281
===========================================================================
Persistent Routes:
  Network Address          Netmask  Gateway Address  Metric
          0.0.0.0          0.0.0.0      192.168.1.1  Default
          0.0.0.0          0.0.0.0      192.168.1.1  Default
===========================================================================

Looks like the problem is

169.254.46.68  255.255.255.255         On-link     169.254.46.68    356

Netmask should have been 255.255.0.0, so that any 169.254.C.D would be carried on with this routing record. So I deleted the faulty line

route -4 delete 169.254.46.68

then issued

route -4 add 169.254.0.0 mask 255.255.0.0 0.0.0.0 metric 3 if 10

10 being my Ethernet adapter as can be seen on the top “route -4 print”. Tricky part for me was to get the gateway was “On-link” in the routing table, after a couple of tries I figured using “0.0.0.0″ as the gateway just makes it “On-link”. After all, whole point of AUTO IP networks is that there’s no router present :)

Now, it works, but after reboots these could be screwed up all over again (despite the fact that route has -p parameter). We’ll see.

Mikrotik RouterOS: How to track each user’s bandwidth usage ?

engin — Fri, 23 Sep 2011 14:36:53 +0000

There are many ways to keep an eye on your users with RouterOS. Two ways to track user bandwidth hoggers:

Use packet sniffer to collect a sample (about a few minutes when there is problem), then examine the data in Connections and Hosts tab.

Pros

Does not consume resources all the time

Cons

Not convenient, you have to do a few clicks and wait for some time to see the result.
You cannot directly act upon the hogger.

Use Firewall rules to match packets for each host and add rate column.

Pros

You can see bandwidth usage of each IP on your network in real time
You can act upon any user and enforce bandwidth limits

Cons

I’ve added ~250 rules to match the traffic for each IP and it caused around ~15% CPU overhead on 300MHz CPU.

As for making all the rules for the firewall. Obviously, I won’t enter those rules by hand, thanks to Mikrotik’s shell it is very easy:

:for i from=2 to=254 do={/ip firewall filter add chain=forward dst-address="193.168.1.$i" action=passthrough}

Passthrough means “do nothing”. We just use the rule to match the packets to each IP then we make use of the rule statistics to get the information we want.

Mysterious performance problems with your good old network

engin — Fri, 23 Sep 2011 14:24:59 +0000

Even though we haven’t changed our network infrastructure for quite some time we started to have network performance problems. Eventually, the network started to stop working from time to time. Router stopped handing out DHCP leases and sometimes freeze all of a sudden.

Not only this, but the networked device which we are developing also started to suffer severely from network problems. The performance was so bad, we couldn’t connect to it over LAN and even if we did, it was a very fragile connection.

We have changed many ADSL Modem routers that you could find on your computer hardware store. None of them could handle it.

Lately, we bought a Mikrotik RB/450. It is a tiny box with 300 MHz CPU and 64MB RAM. With RouterOS installed on it, which is a Linux based solution. I put our ADSL Modem in Bridge Mode, than used the PPPoE client on the RouterOS to connect to the internet. This way, there is basically no load on the ADSL Modem. And voila! Network is performing fantastically.

Couple of observations:

With around 30 active clients, you get 250-600 active connections in NAT table at any given time.
Our monthly bandwidth usage is around 190 GB

But why did this started to happen now ?

The reason the routers started falling is probably the evolving technology and internet using habits of users. Browsers are using more and more connections and users are opening more and more tabs. There this web site which claims listing how many simultaneous connections can a router handle. I haven’t checked reliability of their measurement methods but if we assume the list is accurate, you’d see there are plenty of routers that can’t handle 250 connections (which is our minimum).

I’m extremely pleased with the Mikrotik RB/450 and I recommended it to any SOHO.

The performance problem of our networked devices is another story however. Since, our PCs and devices are connected to the same switch, our connection does not even require a router to orchestrate the packets. Only problem a router could cause trouble for us is the DHCP problem, which can be worked around with assigning static IPs.

However, I started to get suspicious about all the broadcast traffic new Windows versions was generating lately. Then, I did some experiments with our networked device which was having serious network problems, I noticed that it works as expected when

It is directly connected to the PC
Only the device and the PC was connected to a switch — and nothing else is connected to the switch.

When I plug the company LAN to the switch the device started performing poorly again because it tried to check if all those broadcast messages was actually something useful or not. The problem is, there’s no hardware filtering available on the device. As an additional challenge, hardware buffer of the Ethernet is implemented as a FIFO, so I can’t read arbitrary points in the buffer. So, I have to copy the whole hardware buffer to user space before I can actually filter on it. Filtering is easy and not a problem but copying the data is. I’ll see if I can optimize this in the near future.

Moral of the day, new protocols that use broadcasting aggressively are not good network citizens.

uIP 1.0 for Windows

engin — Tue, 12 Jul 2011 09:34:56 +0000

Thanks to some other guy’s (Higepon, I assume) efforts (whom site is now offline), I got uIP-1.0 to run on my Windows 7. You can see and get the code yourself https://github.com/engina/uip-1.0-win

Engin@Engin-VAIO ~/Code/uip-1.0-win/x86-gcc
$ ./uip
0 - (TAP-Win32 Adapter V9)
         fe80::b89d:19c6:5f43:ec4e%32 [0]
         0.0.0.0 [0]
1 - (VMware Virtual Ethernet Adapter)
         fe80::b98e:1fe8:9121:d0f2%24 [0]
         192.168.67.1 [0]
2 - (Microsoft)
         fe80::b4cb:2e7f:d890:9004%21 [0]
         0.0.0.0 [0]
3 - (Microsoft)
         192.168.1.223 [0]
4 - (Microsoft)
         fe80::cd43:876e:3b0b:5b79%14 [0]
         0.0.0.0 [0]
5 - (Microsoft)
         fe80::2daf:5bd7:5b32:618e%15 [0]
         192.168.1.111 [0]
6 - (VMware Virtual Ethernet Adapter)
         fe80::5431:7f2f:1920:632c%23 [0]
         192.168.9.1 [0]
7 - (Intel(R) 82567LM Gigabit Network Connection)
         fe80::5872:faab:8d21:2e44%10 [0]
         192.168.1.142 [0]
Choose:

This is useful as a debugging aid, a reference implementation and rapid development.

Importing something of a bit of a value for personal use in Turkey — part 3 finale

engin — Thu, 07 Apr 2011 06:28:22 +0000

In the third installment of the series :) I have managed to get my package in my third visit to customs. This time everything took 2 hours I paid another €23 (for storage I think). OK, it boils down to this;

1. Ordered items cost: € 524.97
2. Tax paid here %18, €94,49
3. Total distance traveled to get it: ~400km which costs €100 for me here.
4. Another €23 (for storage I assume)
5. Other expenses €20 (see part 2)

Conclusion:

Ordered item: €524
Total expenses: €237 (%45 of item cost)

Anyway, I got all my stuff, including two sets of 8G RAM (one set for my vaio, one set for my macbook) and a C300 256G SSD.

I must tell you that, at first I was not impressed with the performance of SSD when I directly cloned my original hard disk to it. It was noticeably faster but not in a pants-dropping way.

Then, I’ve installed Windows 7 on it. I must say that it is quite fast, i.e. it cold starts Adobe Fireworks in 3 seconds, and hot starts in 1 second. Of course these are not scientific measurements, as I don’t care about those very much either. The perceived speed determines the user experience. So far, it is quite good. Very snappy. We can attribute some of these perforamnce increase to MS technologies such as pre-fetch and superfetch. Oh well…

MySQL Workbench fetching… problem solved

engin — Wed, 06 Apr 2011 20:55:22 +0000

In my previous post about Restoring MySQL Databases from an HDD Image I’ve talked about restoring previous a MySQL development environment from a backup. First of all, there’s a documentation about it, which admittedly I haven’t read. I have simply copied @@datadir and my applications apparently worked fine but Workbench didn’t like it very much. Whenever I tried to expand database tables it kept telling me “Fetching…”. I checked the error logs and it looks something like this:

110406 23:09:00 [Note] Plugin 'FEDERATED' is disabled.
110406 23:09:00 InnoDB: The InnoDB memory heap is disabled
110406 23:09:00 InnoDB: Mutexes and rw_locks use Windows interlocked functions
110406 23:09:00 InnoDB: Compressed tables use zlib 1.2.3
110406 23:09:01 InnoDB: Initializing buffer pool, size = 128.0M
110406 23:09:01 InnoDB: Completed initialization of buffer pool
110406 23:09:01 InnoDB: highest supported file format is Barracuda.
110406 23:09:01  InnoDB: Waiting for the background threads to start
110406 23:09:02 InnoDB: 1.1.5 started; log sequence number 3061735
110406 23:09:02 [ERROR] Missing system table mysql.proxies_priv; please run mysql_upgrade to create it
110406 23:09:02 [ERROR] Native table 'performance_schema'.'events_waits_current' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'events_waits_history' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'events_waits_history_long' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'setup_consumers' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'setup_instruments' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'setup_timers' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'performance_timers' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'threads' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'events_waits_summary_by_thread_by_event_name' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'events_waits_summary_by_instance' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'events_waits_summary_global_by_event_name' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'file_summary_by_event_name' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'file_summary_by_instance' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'mutex_instances' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'rwlock_instances' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'cond_instances' has the wrong structure
110406 23:09:02 [ERROR] Native table 'performance_schema'.'file_instances' has the wrong structure
110406 23:09:02 [Note] Event Scheduler: Loaded 0 events
110406 23:09:02 [Note] C:\PROGRA~2\EASYPH~1.0\MySql\bin\mysqld.exe: ready for connections.
Version: '5.5.10-log'  socket: ''  port: 3306  MySQL Community Server (GPL)
110406 23:13:36 [ERROR] Incorrect definition of table mysql.proc: expected column 'comment' at position 15 to have type text, found type char(64).

If you look careful enough you’ll see please run mysql_upgrade to create it . So I’ve just did it and viola! It looks like fixed. Everything is working fine.