Friday, March 3, 2017

Full Bandwidth Management Parent Queue Tree

Full Bandwidth Management Parent Queue Tree


Full Bandwidth Management Parent Queue Tree
This is the combination, continued and completion of bandwidth management methods previously using parent queue tree about to handle all sorts of purposes in an internet network that is more complex. Perhaps if you do not want to think too hard, you simply apply the simple queue to limit bandwidth that so powerful and assertive to limit bandwidth per client on your internet network. The problem is if we share the bandwidth with a simple queue is stiffness, and its hard for me to divide into many clients normally. It would be appropriate to be applied to the kind of dedicated internet connection.

For example If I have a totally the bandwidth internet connection is up to 2M, after i have tested by speed test, the bandwidth average is 512 kbps. It means that if I have 6 clients, be 512/6 = 86 kbps/clients normally, the ideal number is very small inappropriate for our client. If the bandwidth of up to 2 M calculated with 2000/6 to be around 334 kbps /clients, it can makes browsing and online game stuck if just only one client there who download and play streaming video, because the client spend almost all the average speed (512kbps) other clients will be especially problematic just for browsing or online game. In this case we need the priority to the fourth packets.

Full Bandwidth Management with the parent queue tree complete the lack of packets connection per client, especially to handle the internet network with many clients. The principle is equally divide the bandwidth to all the clients with bandwidth greater than that should be given as the limit bandwidth on each clients, in which we set Online Games, Browsing, Download, Video Streaming packets, connection packets. Well, for now I will make the Full Bandwidth Management using the parent queue tree. For more easily to make the complex rule, I will work with the scripts for more quickly.

Condition Starting:
Total Bandwidth : Up to 2 M

I have 8 Clients
Billing    : 192.168.1.11
Client1    : 192.168.1.17
Client2    : 192.168.1.16
Client3    : 192.168.1.15
Client4    : 192.168.1.14
Client5    : 192.168.1.20
Client6    : 192.168.1.21
Master     : 192.168.1.8

In-Interface   : ether1
Out-Interface  : wlan1

Address List
/ip firewall address-list
add address=192.168.1.0/24  disabled=no list=MikroTik comment=""

1. Separating & Marking Connection Packets "Out & In" Globally

Marking all the connection packet out & in of the traffic mikrotik router, it is relatively depending on the interface that used. Connection packets out from local interface is connection packet In from public interface, it is download connection packet. And Out from local interface is In from public interface, It is upload connection packet. This is implemented on the script below!
/ip firewall mangle add action=mark-connection chain=prerouting disabled=no in-interface=ether1 new-connection-mark=All-Inconn passthrough=yes comment="UPSTEAM CONNECTION" add action=mark-packet chain=prerouting connection-mark=all-inconn disabled=no new-packet-mark=All-inpkt passthrough=yes comment="UPSTEAM" add action=mark-connection chain=forward disabled=no in-interface=wlan1 new-connection-mark=All-Outconn passthrough=yes comment="DOWNSTEAM CONNECTION" add action=mark-packet chain=forward connection-mark=all-outconn disabled=no new-packet-mark=All-Outpkt passthrough=yes comment="DOWNSTEAM" /queue tree add name=All-Bandwidth parent=global-out packet-mark=All-Outpkt queue=default priority=8 max-limit=2M 

We will focus on All-Outconn as you can see as the above scripts, the connection packets using chain=forward and out-interface=wlan1. From this marking we get All-Outpkt that we can separate again to the various kind of connection packets that we need.

2. Separating Download Connection Packets per Client IP 

The general download packet All-Outpkt, we are going to separate it into connection packets per client IP address, to capture the connection download that used by the clients. It is implemented on the following scripts :
/ip firewall mangle add action=mark-packet chain=forward dst-address=192.168.1.11 packet-mark=All-Outpkt new-packet-mark=Billing-pkt passthrough=yes comment="BILLING DOWNSTREAM" add action=mark-packet chain=forward dst-address=192.168.1.17 packet-mark=All-Outpkt new-packet-mark=Client1-pkt passthrough=yes comment="CLIENT1 DOWNSTREAM" add action=mark-packet chain=forward dst-address=192.168.1.16 packet-mark=All-Outpkt new-packet-mark=Client2-pkt passthrough=yes comment="CLIENT2 DOWNSTREAM" add action=mark-packet chain=forward dst-address=192.168.1.15 packet-mark=All-Outpkt new-packet-mark=Client3-pkt passthrough=yes comment="CLIENT3 DOWNSTREAM" add action=mark-packet chain=forward dst-address=192.168.1.14 packet-mark=All-Outpkt new-packet-mark=Client4-pkt passthrough=yes comment="CLIENT4 DOWNSTREAM" add action=mark-packet chain=forward dst-address=192.168.1.20 packet-mark=All-Outpkt new-packet-mark=Client5-pkt passthrough=yes comment="CLIENT5 DOWNSTREAM" add action=mark-packet chain=forward dst-address=192.168.1.21 packet-mark=All-Outpkt new-packet-mark=Client6-pkt passthrough=yes comment="CLIENT6 DOWNSTREAM" add action=mark-packet chain=forward dst-address=192.168.1.8 packet-mark=All-Outpkt new-packet-mark=Master-pkt passthrough=yes comment="MASTER DOWNSTREAM" /queue tree add name=Billing parent=All-Bandwidth packet-mark=Billing-pkt queue=default priority=8 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name=Client1 parent=All-Bandwidth packet-mark=Client1-pkt queue=default priority=8 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name=Client2 parent=All-Bandwidth packet-mark=Client2-pkt queue=default priority=8 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name=Client3 parent=All-Bandwidth packet-mark=Client3-pkt queue=default priority=8 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name=Client4 parent=All-Bandwidth packet-mark=Client4-pkt queue=default priority=8 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name=Client5 parent=All-Bandwidth packet-mark=Client5-pkt queue=default priority=8 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name=Client6 parent=All-Bandwidth packet-mark=Client6-pkt queue=default priority=8 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name=Master parent=All-Bandwidth packet-mark=Master-pkt queue=default priority=8 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s 

The mangle scripts above included with the queue tree rules. From here we have just made the download connections packets per clients, such as
  • Connection Packets Per Client : Billing-pkt, Client1-pkt, Client1-pkt, Client2-pkt, Client3-pkt, Client4-pkt, Client5-pkt, Client6-pkt, Master-pkt

3. Separating Download Packets Clients to Various Kind Connection Packets

The Connections packets per client that we have just made will divided into four kind connection packets such as browsing, online games, download, video streaming. We are going to make 4 kind of the connection packets. If you want to make the kind of connection packets else, you can define using layer 7 protocols.

Part I : Creating the connection packets per client of download and setup the priority in queue tree rules as you wish, you can see the scripts below!
/ip firewall layer7-protocol add comment="" name=download regexp="^.*get.+.(exe|rar|iso|zip|7zip|flv|mkv|avi|mp4|3gp|rmvb|mp3|img|dat|mov).*$" /ip firewall mangle add chain=forward layer7-protocol=download action=mark-packet new-packet-mark=billing-dpkt passthrough=no packet-mark=Billing-pkt comment=Billing-Down add chain=forward layer7-protocol=download action=mark-packet new-packet-mark=client1-dpkt passthrough=no packet-mark=Client1-pkt comment=Client1-Down add chain=forward layer7-protocol=download action=mark-packet new-packet-mark=client2-dpkt passthrough=no packet-mark=Client2-pkt comment=Client2-Down add chain=forward layer7-protocol=download action=mark-packet new-packet-mark=client3-dpkt passthrough=no packet-mark=Client3-pkt comment=Client3-Down add chain=forward layer7-protocol=download action=mark-packet new-packet-mark=client4-dpkt passthrough=no packet-mark=Client4-pkt comment=Client4-Down add chain=forward layer7-protocol=download action=mark-packet new-packet-mark=client5-dpkt passthrough=no packet-mark=Client5-pkt comment=Client5-Down add chain=forward layer7-protocol=download action=mark-packet new-packet-mark=client6-dpkt passthrough=no packet-mark=Client6-pkt comment=Client6-Down add chain=forward layer7-protocol=download action=mark-packet new-packet-mark=master-dpkt passthrough=no packet-mark=Master-pkt comment=Master-Down /ip firewall mangle add action=mark-packet chain=forward connection-bytes=1000000-0 src-port=80,443 passthrough=yes new-packet-mark=billing-dpkt packet-mark=Billing-pkt protocol=tcp comment="Billing-Down" add action=mark-packet chain=forward connection-bytes=1000000-0 src-port=80,443 passthrough=yes new-packet-mark=client1-dpkt packet-mark=Client1-pkt protocol=tcp comment="Billing-Down" add action=mark-packet chain=forward connection-bytes=1000000-0 src-port=80,443 passthrough=yes new-packet-mark=client2-dpkt packet-mark=Client1-pkt protocol=tcp comment="Billing-Down" add action=mark-packet chain=forward connection-bytes=1000000-0 src-port=80,443 passthrough=yes new-packet-mark=client3-dpkt packet-mark=Client1-pkt protocol=tcp comment="Billing-Down" add action=mark-packet chain=forward connection-bytes=1000000-0 src-port=80,443 passthrough=yes new-packet-mark=client4-dpkt packet-mark=Client1-pkt protocol=tcp comment="Billing-Down" add action=mark-packet chain=forward connection-bytes=1000000-0 src-port=80,443 passthrough=yes new-packet-mark=client5-dpkt packet-mark=Client1-pkt protocol=tcp comment="Billing-Down" add action=mark-packet chain=forward connection-bytes=1000000-0 src-port=80,443 passthrough=yes new-packet-mark=client6-dpkt packet-mark=Client1-pkt protocol=tcp comment="Billing-Down" add action=mark-packet chain=forward connection-bytes=1000000-0 src-port=80,443 passthrough=yes new-packet-mark=master-dpkt packet-mark=Master-pkt protocol=tcp comment="Billing-Down" /queue tree add name=Billing-Down parent=Billing packet-mark=billing-dpkt queue=default priority=6 limit-at=180k max-limit=180k burst-limit=256k burst-threshold=135k burst-time=2s add name=Client1-Down parent=Client1 packet-mark=client1-dpkt queue=default priority=6 limit-at=180k max-limit=180k burst-limit=256k burst-threshold=135k burst-time=2s add name=Client2-Down parent=Client2 packet-mark=client2-dpkt queue=default priority=6 limit-at=180k max-limit=180k burst-limit=256k burst-threshold=135k burst-time=2s add name=Client3-Down parent=Client3 packet-mark=client3-dpkt queue=default priority=6 limit-at=180k max-limit=180k burst-limit=256k burst-threshold=135k burst-time=2s add name=Client4-Down parent=Client4 packet-mark=client4-dpkt queue=default priority=6 limit-at=180k max-limit=180k burst-limit=256k burst-threshold=135k burst-time=2s add name=Client5-Down parent=Client5 packet-mark=client5-dpkt queue=default priority=6 limit-at=180k max-limit=180k burst-limit=256k burst-threshold=135k burst-time=2s add name=Client6-Down parent=Client6 packet-mark=client6-dpkt queue=default priority=6 limit-at=180k max-limit=180k burst-limit=256k burst-threshold=135k burst-time=2s add name=Master-Down parent=Master packet-mark=master-dpkt queue=default priority=6 limit-at=180k max-limit=180k burst-limit=256k burst-threshold=135k burst-time=2s 

Here we have used the connections packet per client and to make the download file extension connection packets, so if there are any the extension files that you want to limit, just add the the extension files else to this layer 7 protocols such the above scripts.

Part II. Creating the video streaming connection packet per client and the queue tree rules that you give the priority as you like. Here the scripts that you can paste to the new terminal window, that make more easier.
/ip firewall layer7-protocol add comment="" name=streaming regexp="^.*get.+.(c.youtube.com|cdn.dailymotion.com|metacafe.com|mccont.com).*$" 

If the above script doesnt works, try the second regexp in the layer 7 protocols such the script below!
/ip firewall layer7-protocol add comment="" name=streaming regexp="videoplayback|video" 

/ip firewall mangle add comment=Billing-Streams chain=forward layer7-protocol=streaming action=mark-packet new-packet-mark=billing-spkt passthrough=no packet-mark=Billing-pkt add comment=Client1-Streams chain=forward layer7-protocol=streaming action=mark-packet new-packet-mark=client1-spkt passthrough=no packet-mark=Client1-pkt add comment=Client2-Streams chain=forward layer7-protocol=streaming action=mark-packet new-packet-mark=client2-spkt passthrough=no packet-mark=Client2-pkt add comment=Client3-Streams chain=forward layer7-protocol=streaming action=mark-packet new-packet-mark=client3-spkt passthrough=no packet-mark=Client3-pkt add comment=Client4-Streams chain=forward layer7-protocol=streaming action=mark-packet new-packet-mark=client4-spkt passthrough=no packet-mark=Client4-pkt add comment=Client5-Streams chain=forward layer7-protocol=streaming action=mark-packet new-packet-mark=client5-spkt passthrough=no packet-mark=Client5-pkt add comment=Client6-Streams chain=forward layer7-protocol=streaming action=mark-packet new-packet-mark=client6-spkt passthrough=no packet-mark=Client6-pkt add comment=Master-Streams chain=forward layer7-protocol=streaming action=mark-packet new-packet-mark=master-spkt passthrough=no packet-mark=Master-pkt /queue tree add name=Billing-Streams parent=Billing packet-mark=billing-spkt queue=default priority=7 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Client1-Streams parent=Client1 packet-mark=client1-spkt queue=default priority=7 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Client2-Streams parent=Client2 packet-mark=client2-spkt queue=default priority=7 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Client3-Streams parent=Client3 packet-mark=client3-spkt queue=default priority=7 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Client4-Streams parent=Client4 packet-mark=client4-spkt queue=default priority=7 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Client5-Streams parent=Client5 packet-mark=client5-spkt queue=default priority=7 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Client6-Streams parent=Client6 packet-mark=client6-spkt queue=default priority=7 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Master-Streams parent=Master packet-mark=master-spkt queue=default priority=7 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s 

I think this is the same method to create the connection packets like the download packets that using layer 7 protocols, that allow you to create more and more the connection packet that you wan along you can understand the regexp language

Part III : Creatingthe Online games connection packets per client together with queue tree rules and the priority of the connection packets, here is the scripts:
/ip firewall mangle add action=mark-packet chain=forward packet-mark=All-Outpkt new-packet-mark=gpkt passthrough=yes protocol=tcp dst-port=5340-5352,6000-6152,10001-10011,14009-14030,18901-18909 comment="Online Game Portal" add action=mark-packet chain=forward packet-mark=All-Outpkt new-packet-mark=gpkt passthrough=yes protocol=tcp dst-port=39190,27780,29000,22100,10009,4300,15001,15002,7341,7451 add action=mark-packet chain=forward packet-mark=All-Outpkt new-packet-mark=gpkt passthrough=yes protocol=tcp dst-port=40000,9300,9400,9700,7342,8005-8010,37466,36567,8822 add action=mark-packet chain=forward packet-mark=All-Outpkt new-packet-mark=gpkt passthrough=yes protocol=tcp dst-port=47611,16666,20000,5105,29000,18901-18909,9015 add action=mark-packet chain=forward packet-mark=All-Outpkt new-packet-mark=gpkt passthrough=yes protocol=udp dst-port=27005,27015 add action=mark-packet chain=forward packet-mark=All-Outpkt new-packet-mark=gpkt passthrough=yes protocol=udp dst-port=27005-27020,13055,7800-7900,12060-12070 add action=mark-packet chain=forward packet-mark=All-Outpkt new-packet-mark=gpkt passthrough=yes protocol=udp dst-port=8005-8010,9068,1293,1479,9401,9600,30000 add action=mark-packet chain=forward packet-mark=All-Outpkt new-packet-mark=gpkt passthrough=yes protocol=udp dst-port=14009-14030,42051-42052,40000-40050,13000-13080 /ip firewall mangle add action=mark-packet chain=forward new-packet-mark=billing-gpkt passthrough=no src-address=192.168.1.11 packet-mark=gpkt comment=Billing-Game add action=mark-packet chain=forward new-packet-mark=client1-gpkt passthrough=no src-address=192.168.1.17 packet-mark=gpkt comment=Client1-Game add action=mark-packet chain=forward new-packet-mark=client2-gpkt passthrough=no src-address=192.168.1.16 packet-mark=gpkt comment=Client2-Game add action=mark-packet chain=forward new-packet-mark=client3-gpkt passthrough=no src-address=192.168.1.15 packet-mark=gpkt comment=Client3-Game add action=mark-packet chain=forward new-packet-mark=client4-gpkt passthrough=no src-address=192.168.1.14 packet-mark=gpkt comment=Client4-Game add action=mark-packet chain=forward new-packet-mark=client5-gpkt passthrough=no src-address=192.168.1.20 packet-mark=gpkt comment=Client5-Game add action=mark-packet chain=forward new-packet-mark=client6-gpkt passthrough=no src-address=192.168.1.21 packet-mark=gpkt comment=Client6-Game add action=mark-packet chain=forward new-packet-mark=master-gpkt passthrough=no src-address=192.168.1.8 packet-mark=gpkt comment=Master-Game /queue tree add name=Billing-Game parent=Billing packet-mark=billing-gpkt queue=default priority=1 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Client1-Game parent=Client1 packet-mark=client1-gpkt queue=default priority=1 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Client2-Game parent=Client2 packet-mark=client2-gpkt queue=default priority=1 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Client3-Game parent=Client3 packet-mark=client3-gpkt queue=default priority=1 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Client4-Game parent=Client4 packet-mark=client4-gpkt queue=default priority=1 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Client5-Game parent=Client5 packet-mark=client5-gpkt queue=default priority=1 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Client6-Game parent=Client6 packet-mark=client6-gpkt queue=default priority=1 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s add name=Master-Game parent=Master packet-mark=master-gpkt queue=default priority=1 limit-at=220k max-limit=220k burst-limit=256k burst-threshold=165k burst-time=2s 

In this case we have create the online-games connection, using the port tcp and udp game that have used by the online games server. Therefore you must the online game ports that usually used. Here I collect the ports to be one before we can separate by IP address of the clients that you have. Then finally we can capture the online games packets in the queue tree rules for the priority and manage the bandwidth for this.

Part IV. Creating the browsing connection packets per client complete with queue tree rules and give you can see on the scripts below :
/ip firewall mangle add action=mark-packet chain=forward connection-bytes=0-1000000 src-port=80,443 passthrough=no packet-mark=Billing-pkt new-packet-mark=billing-bpkt protocol=tcp comment="BILLING BROWSING" add action=mark-packet chain=forward connection-bytes=0-1000000 src-port=80,443 passthrough=no packet-mark=Client1-pkt new-packet-mark=client1-bpkt protocol=tcp comment="CLIENT1 BROWSING" add action=mark-packet chain=forward connection-bytes=0-1000000 src-port=80,443 passthrough=no packet-mark=Client2-pkt new-packet-mark=client2-bpkt protocol=tcp comment="CLIENT2 BROWSING" add action=mark-packet chain=forward connection-bytes=0-1000000 src-port=80,443 passthrough=no packet-mark=Client3-pkt new-packet-mark=client3-bpkt protocol=tcp comment="CLIENT3 BROWSING" add action=mark-packet chain=forward connection-bytes=0-1000000 src-port=80,443 passthrough=no packet-mark=Client4-pkt new-packet-mark=client4-bpkt protocol=tcp comment="CLIENT4 BROWSING" add action=mark-packet chain=forward connection-bytes=0-1000000 src-port=80,443 passthrough=no packet-mark=Client5-pkt new-packet-mark=client5-bpkt protocol=tcp comment="CLIENT5 BROWSING" add action=mark-packet chain=forward connection-bytes=0-1000000 src-port=80,443 passthrough=no packet-mark=Client6-pkt new-packet-mark=client6-bpkt protocol=tcp comment="CLIENT6 BROWSING" add action=mark-packet chain=forward connection-bytes=0-1000000 src-port=80,443 passthrough=no packet-mark=Master-pkt new-packet-mark=master-bpkt protocol=tcp comment="MASTER BROWSING" /queue tree add name="Billing-Browsing" parent=Billing packet-mark=billing-bpkt queue=default priority=2 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name="Client1-Browsing" parent=Client1 packet-mark=client1-bpkt queue=default priority=2 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name="Client2-Browsing" parent=Client2 packet-mark=client2-bpkt queue=default priority=2 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name="Client3-Browsing" parent=Client3 packet-mark=client3-bpkt queue=default priority=2 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name="Client4-Browsing" parent=Client4 packet-mark=client4-bpkt queue=default priority=2 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name="Client5-Browsing" parent=Client5 packet-mark=client5-bpkt queue=default priority=2 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name="Client6-Browsing" parent=Client6 packet-mark=client6-bpkt queue=default priority=2 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s add name="Master-Browsing" parent=Master packet-mark=master-bpkt queue=default priority=2 limit-at=256k max-limit=256k burst-limit=720k burst-threshold=192k burst-time=2s 

We still use the connection packets per client to make the browsing connection packets. Here we use the port 80 and 443 (HTTP and HTTPS), we define small bytes connection-bytes=0-1000000, that using the ports usually used when the client browse any url of the website.


Here we have just implemented "Full Bandwidth Management Parent Queue Tree", then you can customize the scripts with the interface of mikrotik router that you have use, the IP Address of the clients that you have, so that you can manage the queue tree rules as the final result. For more quickly you just need to copy all the scripts to the new terminal window of the winbox. So this is the connection Packets with divided into four main of the connection packets such as download, browsing, video streaming, and online games), the queue tree rules will capture the marking that we have created on the mangle rules and the queue tree will manage that packet mark. The result of the experiment method as shown like the tree pictures above! For more details you can see this article video that I have included!

Available link for download