Task #10953
closed
Check accounting-service
Added by Roberto Cirillo over 7 years ago. Updated over 7 years ago.
100%
Description
A set of preproduction nodes have been upgraded in order to send the accounting records to accounting-service instance and not directly to couchbase cluster.
For the moment, the accounting instance is the following:
accounting-service-d.d4science.org
Please could you check if the records are correctly send and managed by accounting service?
Files
| accounting-data-simple.txt (541 Bytes) accounting-data-simple.txt | Luca Frosini, Jan 17, 2018 12:38 PM | ||
| accounting-data.txt (3.85 KB) accounting-data.txt | Luca Frosini, Jan 17, 2018 12:38 PM |
Related issues
Updated by Roberto Cirillo over 7 years ago
- Related to Upgrade #10933: /gcube/preprod upgrade to gCube 4.10.0 added
Updated by Luca Frosini over 7 years ago
The stress test of the service succeeded.
The first tests was made with 10 parallels processes sending each one 100.000 records to the service (one by one).
The second tests was made with 10 subsequent processes sending each one 10.000 records to the service in bulk operation (please note that this situation is not possibile).
the tests were made with debug level of receiving Servlet to TRACE LEVEL to stress also the disk i/o.
The records of stress test could be aggregated.
The records were saved on fallback file because it seems that the document-store-lib-couchbase-latest.jar has not been found.
Maybe is a problems of rights. All the other libraries have
-rwxr-xr-x
instead document-store-lib-couchbase-latest.jar has (no executable permission)
-rw-r--r--
I'm going to change the permission manually and restarting the container.
Please note that the fallback file is recreated correctly (with no recursion) so it seems also that the bug of exponentially growing fallbacks seems fixed.
Updated by Luca Frosini over 7 years ago
- File accounting-data.txt accounting-data.txt added
- File accounting-data-simple.txt accounting-data-simple.txt added
I also used ab for stress test. You can test it by your own by using :
ab -n 100000 -c 100 -T 'application/json' -p accounting-data-simple.txt -H 'gcube-token: YOUR_DEV_OR_PREPROD_TOKEN_HERE' https://accounting-service-d.d4science.org:443/accounting-service/record
My result of benchmark is:
$ab -n 100000 -c 100 -T 'application/json' -p accounting-data-simple.txt -H 'gcube-token: YOUR_DEV_OR_PREPROD_TOKEN_HERE' https://accounting-service-d.d4science.org:443/accounting-service/record
This is ApacheBench, Version 2.3 <$Revision: 1796539 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking accounting-service-d.d4science.org (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests
Server Software: nginx
Server Hostname: accounting-service-d.d4science.org
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-ECDSA-AES128-GCM-SHA256,256,128
TLS Server Name: accounting-service-d.d4science.org
Document Path: /accounting-service/record
Document Length: 0 bytes
Concurrency Level: 100
Time taken for tests: 99.082 seconds
Complete requests: 100000
Failed requests: 0
Total transferred: 53600000 bytes
Total body sent: 78200000
HTML transferred: 0 bytes
Requests per second: 1009.27 [#/sec] (mean)
Time per request: 99.082 [ms] (mean)
Time per request: 0.991 [ms] (mean, across all concurrent requests)
Transfer rate: 528.29 [Kbytes/sec] received
770.75 kb/s sent
1299.04 kb/s total
Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 31 34.0 26 1086
Processing: 1 68 55.3 51 616
Waiting: 1 68 55.2 51 488
Total: 4 99 60.7 86 1306
Percentage of the requests served within a certain time (ms)
50% 86
66% 102
75% 117
80% 128
90% 169
95% 211
98% 265
99% 301
100% 1306 (longest request)
or
ab -n 100000 -c 100 -T 'application/json' -p accounting-data.txt -H 'gcube-token: YOUR_DEV_OR_PREPROD_TOKEN_HERE' https://accounting-service-d.d4science.org:443/accounting-service/record
My result of benchmark is:
$ ab -n 100000 -c 100 -T 'application/json' -p accounting-data.txt -H 'gcube-token: YOUR_DEV_OR_PREPROD_TOKEN_HERE' https://accounting-service-d.d4science.org:443/accounting-service/record
This is ApacheBench, Version 2.3 <$Revision: 1796539 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking accounting-service-d.d4science.org (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests
Server Software: nginx
Server Hostname: accounting-service-d.d4science.org
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-ECDSA-AES128-GCM-SHA256,256,128
TLS Server Name: accounting-service-d.d4science.org
Document Path: /accounting-service/record
Document Length: 0 bytes
Concurrency Level: 100
Time taken for tests: 185.942 seconds
Complete requests: 100000
Failed requests: 0
Total transferred: 53600000 bytes
Total body sent: 418900000
HTML transferred: 0 bytes
Requests per second: 537.80 [#/sec] (mean)
Time per request: 185.942 [ms] (mean)
Time per request: 1.859 [ms] (mean, across all concurrent requests)
Transfer rate: 281.51 [Kbytes/sec] received
2200.05 kb/s sent
2481.56 kb/s total
Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 15 46.8 5 3035
Processing: 2 171 156.5 132 1485
Waiting: 2 171 156.3 132 1485
Total: 4 186 163.8 144 3285
Percentage of the requests served within a certain time (ms)
50% 144
66% 216
75% 267
80% 302
90% 403
95% 498
98% 625
99% 720
Please find attached the requested files.
Updated by Luca Frosini over 7 years ago
- Assignee changed from Luca Frosini to Roberto Cirillo
The records are stored in fallback because document-store-lib-couchbase does not have the required dependencies.
You can use the jar-with-dependecies or add also java-client-2.2.7.jar and rxjava-1.0.17.jar to tomcat/lib.
Updated by Roberto Cirillo over 7 years ago
- Assignee changed from Roberto Cirillo to Luca Frosini
Done. Could you recheck please?
Updated by Luca Frosini over 7 years ago
- Status changed from In Progress to Feedback
- % Done changed from 0 to 100
It seems working correctly now persisting the records on couchbase.
I also run the ab stress tests again and they succeeded.
