getBestRegionHost fails with ArrayIndexOutOfBoundsException

[thirdeyenick]

Hi,
we tried splicer today. After building it and setting up the correct firewall permissions for our regionservers it sadly fails when doing a test query:

curl 'http://127.0.0.1:4245/api/query' --data-binary '{"start":1473715620025,"queries":[{"metric":"nine.telegraf.cpu_usage_user","aggregator":"sum","rate":false,"rateOptions":{"counter":false},"downsample":"10m-max","tags":{}}]}' {"error": "java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException, stacktrace=com.turn.splicer.tsdbutils.SplicerQueryRunner.runQuerySlices(SplicerQueryRunner.java:110), com.turn.splicer.tsdbutils.SplicerQueryRunner.sliceAndRunQuery(SplicerQueryRunner.java:61)"}

I looks like it fails in the end with an "ArrayIndexOutOfBoundsException" in the getBestRegionHost Function:

Oct 05 17:08:48 metrics-splicer-cz41-1 java[12804]: Caused by: java.lang.ArrayIndexOutOfBoundsException: null Oct 05 17:08:48 metrics-splicer-cz41-1 java[12804]: at java.lang.System.arraycopy(Native Method) ~[na:1.8.0_91] Oct 05 17:08:48 metrics-splicer-cz41-1 java[12804]: at com.turn.splicer.hbase.RegionChecker.getBestRegionHost(RegionChecker.java:62) ~[tsdb-splicer-all-0.1.5-2016011901.jar:na] Oct 05 17:08:48 metrics-splicer-cz41-1 java[12804]: at com.turn.splicer.HttpWorker.call(HttpWorker.java:71) ~[tsdb-splicer-all-0.1.5-2016011901.jar:na] Oct 05 17:08:48 metrics-splicer-cz41-1 java[12804]: at com.turn.splicer.HttpWorker.call(HttpWorker.java:27) ~[tsdb-splicer-all-0.1.5-2016011901.jar:na] Oct 05 17:08:48 metrics-splicer-cz41-1 java[12804]: at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_91] Oct 05 17:08:48 metrics-splicer-cz41-1 java[12804]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_91] Oct 05 17:08:48 metrics-splicer-cz41-1 java[12804]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_91]

The correspnding MetricsCacher seems to get a result though:

Oct 05 17:08:48 metrics-splicer-cz41-1 java[12804]: 2016-10-05 17:08:48,069 INFO [splice-pool-1-thread-0] MetricsCache: Looking up key for metric=nine.telegraf.cpu_usage_user. Found result=[0, 0, 35]

Are we doing anything wrong? Thanks for any help.

[rlex]

Same here, sadly.

2016-12-21_12:48:58.92161 2016-12-21 12:48:58,921 INFO  [jetty-qt-pool-14] SuggestHttpWorker: Sending query=http://localhost:4242/api/suggest?max=1000&q=agent_violet.app.connected_cameras&type=metrics to TSD running on host=localhost
2016-12-21_12:48:58.98441 2016-12-21 12:48:58,984 INFO  [jetty-qt-pool-14] SuggestHttpWorker: Result=["agent_violet.app.connected_cameras"]
2016-12-21_12:48:58.98540 2016-12-21 12:48:58,985 INFO  [jetty-qt-pool-14] SuggestHttpWorker: Returned localhost:4242 into the available queue
2016-12-21_12:48:59.21815 2016-12-21 12:48:59,217 INFO  [jetty-qt-pool-13] SuggestHttpWorker: Sending query=http://localhost:4242/api/suggest?max=1000&q=&type=metrics to TSD running on host=localhost
2016-12-21_12:48:59.23335 2016-12-21 12:48:59,233 INFO  [jetty-qt-pool-13] SuggestHttpWorker: Result=["agent_violet.app.connected_cameras","agent_violet.app.num_goroutine","agent_violet.gc.pause.median","agent_violet.memstat.alloc","agent_violet.memstat.sys","agent_violet.pause.median","app.vt_preview.accept","app.vt_preview.accept.diff","app.vt_preview.accept.now","app.vt_preview.done","app.vt_preview.done.diff","app.vt_preview.done.now","app.vt_preview.error","app.vt_preview.error.now","app.vt_preview.error_no_target","app.vt_preview.error_no_target.now","app.vt_preview.failed","app.vt_preview.failed.diff","app.vt_preview.failed.now","app.vt_preview.received_message","app.vt_preview.received_message.diff","app.vt_preview.received_message.now","app.vt_preview.received_message_send_to_worker","app.vt_preview.received_message_send_to_worker.diff","app.vt_preview.received_message_send_to_worker.now","app.vt_preview.reject","app.vt_preview.reject.diff","app.vt_preview.reject.now","baker.queue.failed","baker.queue.length","livescreens.stat","nginx.request.count","nginx.request.sla","opentsdb.tsd.rpc.errors","opentsdb.tsd.rpc.exceptions","opentsdb.tsd.rpc.received","pgsql.connections","pgsql.locks","pgsql.replication.lag","pgsql.reserved.connections","pgsql.transactions","pipelinemon.stat","playout.workers","playout.workers.running","runit.service.cpu","runit.service.memory","runit.service.uptime","sidekiq.jobs.busy","sidekiq.jobs.enqueued","sidekiq.jobs.failed","stalin.gc.gc_per_second","stalin.gc.pause","stalin.gc.pause_per_second","stalin.goroutines","stalin.memory.allocated","stalin.memory.frees","stalin.memory.heap","stalin.memory.mallocs","stalin.memory.stack","sync.status","sys.bcache.cache_hit_ratio","sys.bcache.cache_hit_ratio.5m","sys.cpu.iowait","sys.cpu.usage","sys.disk.free","sys.disk.iops","sys.disk.percent_inode","sys.disk.percent_usage","sys.disk.total","sys.la.one_min","sys.memory.buffers","sys.memory.cached","sys.memory.free","sys.memory.free_bc","sys.memory.percent_free","sys.memory.percent_swap","sys.memory.total","sys.memory.used","sys.net.bytes","sys.net.drop","sys.net.errs","sys.net.tcp.in_errs","sys.net.tcp.retrans_segs","sys.net.udp.in_errors","sys.net.udp.receive_buf_errors","sys.net.udp.send_buf_errors","sys.openfiles","sys.proc.blocked","sys.proc.defunct","sys.proc.fork_rate","sys.proc.running","varnish.stat.cache","varnish.stat.connections"]
2016-12-21_12:48:59.23388 2016-12-21 12:48:59,233 INFO  [jetty-qt-pool-13] SuggestHttpWorker: Returned localhost:4242 into the available queue
2016-12-21_12:49:01.76871 2016-12-21 12:49:01,768 INFO  [jetty-qt-pool-13] SuggestHttpWorker: Sending query=http://localhost:4242/api/suggest?max=1000&q=&type=metrics to TSD running on host=localhost
2016-12-21_12:49:01.79487 2016-12-21 12:49:01,794 INFO  [jetty-qt-pool-13] SuggestHttpWorker: Result=["agent_violet.app.connected_cameras","agent_violet.app.num_goroutine","agent_violet.gc.pause.median","agent_violet.memstat.alloc","agent_violet.memstat.sys","agent_violet.pause.median","app.vt_preview.accept","app.vt_preview.accept.diff","app.vt_preview.accept.now","app.vt_preview.done","app.vt_preview.done.diff","app.vt_preview.done.now","app.vt_preview.error","app.vt_preview.error.now","app.vt_preview.error_no_target","app.vt_preview.error_no_target.now","app.vt_preview.failed","app.vt_preview.failed.diff","app.vt_preview.failed.now","app.vt_preview.received_message","app.vt_preview.received_message.diff","app.vt_preview.received_message.now","app.vt_preview.received_message_send_to_worker","app.vt_preview.received_message_send_to_worker.diff","app.vt_preview.received_message_send_to_worker.now","app.vt_preview.reject","app.vt_preview.reject.diff","app.vt_preview.reject.now","baker.queue.failed","baker.queue.length","livescreens.stat","nginx.request.count","nginx.request.sla","opentsdb.tsd.rpc.errors","opentsdb.tsd.rpc.exceptions","opentsdb.tsd.rpc.received","pgsql.connections","pgsql.locks","pgsql.replication.lag","pgsql.reserved.connections","pgsql.transactions","pipelinemon.stat","playout.workers","playout.workers.running","runit.service.cpu","runit.service.memory","runit.service.uptime","sidekiq.jobs.busy","sidekiq.jobs.enqueued","sidekiq.jobs.failed","stalin.gc.gc_per_second","stalin.gc.pause","stalin.gc.pause_per_second","stalin.goroutines","stalin.memory.allocated","stalin.memory.frees","stalin.memory.heap","stalin.memory.mallocs","stalin.memory.stack","sync.status","sys.bcache.cache_hit_ratio","sys.bcache.cache_hit_ratio.5m","sys.cpu.iowait","sys.cpu.usage","sys.disk.free","sys.disk.iops","sys.disk.percent_inode","sys.disk.percent_usage","sys.disk.total","sys.la.one_min","sys.memory.buffers","sys.memory.cached","sys.memory.free","sys.memory.free_bc","sys.memory.percent_free","sys.memory.percent_swap","sys.memory.total","sys.memory.used","sys.net.bytes","sys.net.drop","sys.net.errs","sys.net.tcp.in_errs","sys.net.tcp.retrans_segs","sys.net.udp.in_errors","sys.net.udp.receive_buf_errors","sys.net.udp.send_buf_errors","sys.openfiles","sys.proc.blocked","sys.proc.defunct","sys.proc.fork_rate","sys.proc.running","varnish.stat.cache","varnish.stat.connections"]
2016-12-21_12:49:01.79558 2016-12-21 12:49:01,795 INFO  [jetty-qt-pool-13] SuggestHttpWorker: Returned localhost:4242 into the available queue
2016-12-21_12:49:03.81892 2016-12-21 12:49:03,818 INFO  [jetty-qt-pool-18] SplicerServlet: Serving query=TsQuery{start='1482302943514', end='null', timezone='null', options=null, padding=false, no_annotations=false, with_global_annotations=true, show_tsuids=false, queries=[TSSubQuery(metric=app.vt_preview.error, tags=[], tsuids=[], agg=sum, downsample=10s-avg, ds_interval=0, rate=false, rate_options=null)], start_time=1482302943514, end_time=1482324543818, ms_resolution=false}
2016-12-21_12:49:03.81978 2016-12-21 12:49:03,819 INFO  [jetty-qt-pool-18] SplicerServlet: Original TsQuery Start time=2016/12/21-06:49:03, End time=2016/12/21-12:49:03
2016-12-21_12:49:03.98805 16/12/21 12:49:03 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x1c4e2514 connecting to ZooKeeper ensemble=riemann-2.production.****:2181
2016-12-21_12:49:04.57743 2016-12-21 12:49:04,577 INFO  [splice-pool-1-thread-0] MetricsCache: Looking up key for metric=app.vt_preview.error. Found result=[0, 0, 92]
2016-12-21_12:49:04.58250 2016-12-21 12:49:04,582 ERROR [jetty-qt-pool-18] SplicerQueryRunner: Could not execute HTTP Queries
2016-12-21_12:49:04.58251 java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException
2016-12-21_12:49:04.58251 	at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.8.0_91]
2016-12-21_12:49:04.58251 	at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[na:1.8.0_91]
2016-12-21_12:49:04.58251 	at com.turn.splicer.tsdbutils.SplicerQueryRunner.runQuerySlices(SplicerQueryRunner.java:93) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58252 	at com.turn.splicer.tsdbutils.SplicerQueryRunner.sliceAndRunQuery(SplicerQueryRunner.java:61) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58252 	at com.turn.splicer.SplicerServlet.doPostWork(SplicerServlet.java:210) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58252 	at com.turn.splicer.SplicerServlet.doPost(SplicerServlet.java:70) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58252 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58252 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58252 	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58253 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58253 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:431) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58253 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58253 	at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58253 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58253 	at org.eclipse.jetty.server.Server.handle(Server.java:370) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58255 	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58255 	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58255 	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58255 	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58255 	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58255 	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58256 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58256 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58256 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58256 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58256 	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
2016-12-21_12:49:04.58256 Caused by: java.lang.ArrayIndexOutOfBoundsException: null
2016-12-21_12:49:04.58256 	at java.lang.System.arraycopy(Native Method) ~[na:1.8.0_91]
2016-12-21_12:49:04.58257 	at com.turn.splicer.hbase.RegionChecker.getBestRegionHost(RegionChecker.java:62) ~[opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58257 	at com.turn.splicer.HttpWorker.call(HttpWorker.java:71) ~[opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58257 	at com.turn.splicer.HttpWorker.call(HttpWorker.java:27) ~[opentsdb_splicer.jar:na]
2016-12-21_12:49:04.58257 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_91]
2016-12-21_12:49:04.58257 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_91]
2016-12-21_12:49:04.58257 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_91]
2016-12-21_12:49:04.58257 	... 1 common frames omitted
2016-12-21_12:49:04.58411 2016-12-21 12:49:04,583 INFO  [jetty-qt-pool-18] SplicerQueryRunner: Shutdown thread pool for query={1482302943514 to 1482324543818, m=[app.vt_preview.error{}, downsample=10s-avg, rate=false, tags={}]}
2016-12-21_12:49:04.58600 16/12/21 12:49:04 INFO client.HConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x158f7f9f861036a
2016-12-21_12:49:04.61898 2016-12-21 12:49:04,618 ERROR [jetty-qt-pool-18] SplicerServlet: Exception which processing POST request
2016-12-21_12:49:04.61900 java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException
2016-12-21_12:49:04.61901 	at com.turn.splicer.tsdbutils.SplicerQueryRunner.runQuerySlices(SplicerQueryRunner.java:110) ~[opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61901 	at com.turn.splicer.tsdbutils.SplicerQueryRunner.sliceAndRunQuery(SplicerQueryRunner.java:61) ~[opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61902 	at com.turn.splicer.SplicerServlet.doPostWork(SplicerServlet.java:210) ~[opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61902 	at com.turn.splicer.SplicerServlet.doPost(SplicerServlet.java:70) ~[opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61902 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61903 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61903 	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61903 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61904 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:431) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61904 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61904 	at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61905 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61905 	at org.eclipse.jetty.server.Server.handle(Server.java:370) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61905 	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61905 	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61905 	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61906 	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61906 	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61909 	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61909 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61909 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61909 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61910 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61910 	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
2016-12-21_12:49:04.61910 Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException
2016-12-21_12:49:04.61910 	at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.8.0_91]
2016-12-21_12:49:04.61910 	at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[na:1.8.0_91]
2016-12-21_12:49:04.61910 	at com.turn.splicer.tsdbutils.SplicerQueryRunner.runQuerySlices(SplicerQueryRunner.java:93) ~[opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61911 	... 23 common frames omitted
2016-12-21_12:49:04.61911 Caused by: java.lang.ArrayIndexOutOfBoundsException: null
2016-12-21_12:49:04.61911 	at java.lang.System.arraycopy(Native Method) ~[na:1.8.0_91]
2016-12-21_12:49:04.61911 	at com.turn.splicer.hbase.RegionChecker.getBestRegionHost(RegionChecker.java:62) ~[opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61911 	at com.turn.splicer.HttpWorker.call(HttpWorker.java:71) ~[opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61911 	at com.turn.splicer.HttpWorker.call(HttpWorker.java:27) ~[opentsdb_splicer.jar:na]
2016-12-21_12:49:04.61911 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_91]
2016-12-21_12:49:04.61911 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_91]
2016-12-21_12:49:04.61911 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_91]
2016-12-21_12:49:04.61912 	... 1 common frames omitted

[supernomad]

Same thing is happening on our systems.

[supernomad]

So anyone with this issue, and I assume thats everyone, needs to change this line https://github.com/turn/splicer/blob/master/src/main/java/com/turn/splicer/hbase/RegionChecker.java#L34, from 4 to 3.

Or no matter what splicer will never work.

[wicknicks]

We use metric width of 4 in our TSDB installation. AFAIK, this is a configurable parameter in TSDB. We could do the same here.

[supernomad]

@wicknicks I am sorry for the wording, I meant anyone with this issue will not get splicer working without the change. Mainly because I wasn't aware that there was a setting in tsdb to change this, I honestly thought it was some versioning thing. That being said I am definitely +1 on the idea to make this configurable, as I can't imagine changing the metric width in tsdb is simple haha.

[wicknicks]

No problem. I will create a pull request with the changes in the few days.

[mancubus77]

Faced same issue, my OpenTSDB has default setting

http://opentsdb.net/docs/build/html/user_guide/configuration.html

Property	Type	Required	Description	Default	CLI
tsd.storage.uid.width.metric (2.2)	Integer	Optional	The width, in bytes, of metric UIDs. WARNING Do not change after writing data to HBase or you will corrupt your tables and not be able to query any more.	3

@supernomad's advise didn't help

I would be appreciated for any advice

[mancubus77]

All good, thanks to gradlew it's fixed. One annoying thing is left, there is no logging anymore.

SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#noProviders for further details.
SLF4J: Class path contains SLF4J bindings targeting slf4j-api versions prior to 1.8.
SLF4J: Ignoring binding found at [jar:file:/usr/share/tsdb-splicer/tsdb-splicer-all-0.1.5-2016011901.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#ignoredBindings for an explanation.

[alugowski]

Regarding logging, if it's not set up in your ops environment you can add a log4j.properties to the resource directory so SLF4J can pick it up.

Example log4j.propertes that will just write to stdout:

log4j.rootLogger=INFO, stdout

# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c:%L - %m%n

[mancubus77]

Thanks @alugowski
Possible something wrong with my build file. I've updated properties and replaced Log provider. No love
Need to spend more time one this.

[wicknicks]

@mancubus77 try adding an additional JVM parameter -Dlog4j.configuration=file:///absolute/path/to/your/mylogging.properties -- maybe the resources directory is not being added to your classpath.