Karan Singh

Where there's a Cloud , there's a way !!

COSBench: Cloud Object Storage Benchmarking Tool

| Comments

cosbench Benchmarking storage systems is a pretty interesting job and these days i am playing hard with it . Having said that object storage benchmarking is currently under my radar.

Block and file storage is there since ancient times and so as their benchmarking tools. Object storage is way too different and the traditional benchmarking tools don’t talk object. There are not very many tools out there for benchmarking object storage, the one which is getting popularity is COSBench aka Cloud Storage Benchmarking tool.

COSbench is developed and open sourced by Intel. It’s based on classic client / server model where client node aka COSBench Driver node ( COSBench client ) accepts the job from server aka COSBench controller node ( COSBench server) executes the job against storage system gathers the result and send them back to COSBench controller node. Here are some good to know things about COSBench

  • COSBench works on RHEL / Debian based machines ( for windows i don’t know )
  • Based on JAVA and OSGi framework
  • Yes you can run COSBench Controller and COSBench driver on a single node
  • Supports a pretty long list of object storage backends / COSBench drivers
  • Submit workloads either from GUI or CLI
  • Result data can be exported as CSV
  • One COSBench controller can connect to multiple COSBench driver nodes

Setting up COSBench on RHEL based nodes

  • Install dependencies

$ sudo yum install -y wget unzip java curl nmap-ncat

  • Note: The standard way of getting open source softwares i.e. git clone might not work at first place , so you need to download binary packages for COSBench
1
2
3
4
$ wget https://github.com/intel-cloud/cosbench/releases/download/v0.4.2.c3/0.4.2.c3.zip
$ unzip 0.4.2.c3.zip
$ ln -s 0.4.2.c3 cos
$ cd cos; chmod +x *.sh
  • Next start COSBench controller and COSBench driver by running $ sh start-all.sh if you are getting warnings like Ncat: Connection refused. you can safely ignore them.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
[[email protected] cos]$ sh start-all.sh
Launching osgi framwork ...
Successfully launched osgi framework!
Booting cosbench driver ...
Ncat: Connection refused.
.Ncat: Connection refused.
...
Starting    cosbench-log_0.4.2    [OK]
.
Starting    cosbench-tomcat_0.4.2    [OK]
Starting    cosbench-config_0.4.2    [OK]
Starting    cosbench-http_0.4.2    [OK]
Starting    cosbench-cdmi-util_0.4.2    [OK]
Starting    cosbench-core_0.4.2    [OK]
Starting    cosbench-core-web_0.4.2    [OK]
Starting    cosbench-api_0.4.2    [OK]
Starting    cosbench-mock_0.4.2    [OK]
Starting    cosbench-ampli_0.4.2    [OK]
Starting    cosbench-swift_0.4.2    [OK]
Starting    cosbench-keystone_0.4.2    [OK]
Starting    cosbench-httpauth_0.4.2    [OK]
Starting    cosbench-s3_0.4.2    [OK]
Starting    cosbench-librados_0.4.2    [OK]
Starting    cosbench-scality_0.4.2    [OK]
Starting    cosbench-cdmi-swift_0.4.2    [OK]
Starting    cosbench-cdmi-base_0.4.2    [OK]
Starting    cosbench-driver_0.4.2    [OK]
Starting    cosbench-driver-web_0.4.2    [OK]
Successfully started cosbench driver!
Listening on port 0.0.0.0/0.0.0.0:18089 ...
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 18088 !!!
----------------------------------------------

======================================================

Launching osgi framwork ...
Successfully launched osgi framework!
Booting cosbench controller ...
Ncat: Connection refused.
.Ncat: Connection refused.
.Ncat: Connection refused.
.Ncat: Connection refused.
....
Starting    cosbench-log_0.4.2    [OK]
..
Starting    cosbench-tomcat_0.4.2    [OK]
Starting    cosbench-config_0.4.2    [OK]
Starting    cosbench-core_0.4.2    [OK]
Starting    cosbench-core-web_0.4.2    [OK]
Starting    cosbench-controller_0.4.2    [OK]
Starting    cosbench-controller-web_0.4.2    [OK]
Successfully started cosbench controller!
Listening on port 0.0.0.0/0.0.0.0:19089 ...
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 19088 !!!
----------------------------------------------
[[email protected] cos]$
  • Verify if COSBench Controller and driver daemons are listening
1
2
3
4
5
6
7
8
[[email protected] cos]$ netstat -plunt | grep -i java
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 0.0.0.0:18088           0.0.0.0:*               LISTEN      32473/java
tcp        0      0 0.0.0.0:18089           0.0.0.0:*               LISTEN      32473/java
tcp        0      0 0.0.0.0:19088           0.0.0.0:*               LISTEN      32627/java
tcp        0      0 0.0.0.0:19089           0.0.0.0:*               LISTEN      32627/java
[[email protected] cos]$
  • Finally shoot a sample workload from CLI $ sh cli.sh submit conf/workload-config.xml

  • For COSBench Controller UI navigate to http://<Node_IP>:19088/controller/index.html

  • For COSBench Driver UI navigate to http://<Node_IP>:18088/driver/index.html

Sample COSBench workload

The workload is injected in the form of XML files , you can either create workload file from GUI or manually create a XML file and submit your job. Here is the example of a COSBench workload file

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
<?xml version="1.0" encoding="UTF-8" ?>
<workload name="s3-sample" description="sample benchmark for s3">
  <storage type="s3" config="accesskey=<accesskey>;secretkey=<scretkey>;proxyhost=<proxyhost>;proxyport=<proxyport>;endpoint=<endpoint>" />
  <workflow>
    <workstage name="init">
      <work type="init" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
    </workstage>
    <workstage name="prepare">
      <work type="prepare" workers="1" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,10);sizes=c(64)KB" />
    </workstage>
    <workstage name="main">
      <work name="main" workers="8" runtime="30">
        <operation type="read" ratio="80" config="cprefix=s3testqwer;containers=u(1,2);objects=u(1,10)" />
        <operation type="write" ratio="20" config="cprefix=s3testqwer;containers=u(1,2);objects=u(11,20);sizes=c(64)KB" />
      </work>
    </workstage>
    <workstage name="cleanup">
      <work type="cleanup" workers="1" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,20)" />
    </workstage>
    <workstage name="dispose">
      <work type="dispose" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
    </workstage>
  </workflow>
</workload>

For more details on COSBench checkout the official documentation @GitHUb

COSBench seems to be a powerful and much needed tool for Object Storage benchmarking. It’s trying to establish itself as a standard tool for this purpose. Well Standards are usually good unless government makes them ;-P

Comments