java - Fail to read from bigtable in dataflow -

-

i using dataflow work write data bigtable.
currently, got task read rows bigtable.
however, whenever try read rows bigtable using bigtable-hbase-dataflow, fails , complains follow.

 error:   (3218070e4dd208d3): java.lang.illegalargumentexception: b <= @ org.apache.hadoop.hbase.util.bytes.iterateonsplits(bytes.java:1720) @ org.apache.hadoop.hbase.util.bytes.split(bytes.java:1683) @ org.apache.hadoop.hbase.util.bytes.split(bytes.java:1664) @ com.google.cloud.bigtable.dataflow.cloudbigtableio$abstractsource.split(cloudbigtableio.java:512) @ com.google.cloud.bigtable.dataflow.cloudbigtableio$abstractsource.getsplits(cloudbigtableio.java:358) @ com.google.cloud.bigtable.dataflow.cloudbigtableio$source.splitintobundles(cloudbigtableio.java:593) @ com.google.cloud.dataflow.sdk.runners.worker.workercustomsources.performsplit(workercustomsources.java:413) @ com.google.cloud.dataflow.sdk.runners.worker.workercustomsources.performsplitwithapilimit(workercustomsources.java:171) @ com.google.cloud.dataflow.sdk.runners.worker.workercustomsources.performsplit(workercustomsources.java:149) @ com.google.cloud.dataflow.sdk.runners.worker.sourceoperationexecutor.execute(sourceoperationexecutor.java:58) @ com.google.cloud.dataflow.sdk.runners.worker.dataflowworker.executework(dataflowworker.java:288) @ com.google.cloud.dataflow.sdk.runners.worker.dataflowworker.dowork(dataflowworker.java:221) @ com.google.cloud.dataflow.sdk.runners.worker.dataflowworker.getandperformwork(dataflowworker.java:173) @ com.google.cloud.dataflow.sdk.runners.worker.dataflowworkerharness$workerthread.dowork(dataflowworkerharness.java:193) @ com.google.cloud.dataflow.sdk.runners.worker.dataflowworkerharness$workerthread.call(dataflowworkerharness.java:173) @ com.google.cloud.dataflow.sdk.runners.worker.dataflowworkerharness$workerthread.call(dataflowworkerharness.java:160) @ java.util.concurrent.futuretask.run(futuretask.java:266) @ java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor.java:1142) @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:617) @ java.lang.thread.run(thread.java:745)

i using 'com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:1.6.0' , 'com.google.cloud.bigtable:bigtable-hbase-dataflow:0.9.0' now.

here's code. 

cloudbigtablescanconfiguration config = new cloudbigtablescanconfiguration.builder()     .withprojectid("project-id")     .withinstanceid("instance-id")     .withtableid("table")     .build(); pipeline.apply(read.<result>from(cloudbigtableio.read(config)))     .apply(pardo.of(new test()));

fyi, read bigtable , count rows using aggregator in test dofn. 

static class test extends dofn<result, result> {     private static final long serialversionuid = 0l;     private final aggregator<long, long> rowcount = createaggregator("row_count", new sum.sumlongfn());      @override     public void processelement(processcontext c) {         rowcount.addvalue(1l);         c.output(c.element());     } }

i followed tutorial on dataflow document, fails. can me out?

the root cause dependency issue:

previously, our build file omitted dependency:

compile 'io.netty:netty-tcnative-boringssl-static:1.1.33.fork22'

today, added dependency , resolved issues. double-checked problem arises when don't have in build file.

from https://github.com/googlecloudplatform/cloud-bigtable-client/issues/912#issuecomment-249999380.


Comments

Post a Comment

Popular posts from this blog

Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12:test (default-test) on project.Error occurred in starting fork -

-
i searched on internet provided answers did not in building project. the error starting fork. tried following methods: mvn clean install -u when searched, people said because of java 7 , should work java 6. using java 6 only. some said set surefire plugin this: <groupid>org.apache.maven.plugins</groupid> <artifactid>maven-surefire-plugin</artifactid> <version>2.5</version> <configuration> <skiptests>false</skiptests> <testfailureignore>true</testfailureignore> <forkmode>once</forkmode> </configuration> the actual error is: [error] failed execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12:test (default-test) on project junitcategorizer.instrument: error occurred in starting fork, check output in log -> [help 1] org.apache.maven.lifecycle.lifecycleexecutionexception: failed execute goal org.apache.maven.plugins:maven-surefire-plugin:2...
Read more

windows - Debug iNetMgr.exe unhandle exception System.Management.Automation.CmdletInvocationException -

-
i fired windows 7 machine today , attempting start intemgr.exe process terminates immediately. started process under debugger. seems process terminating following exception: system.management.automation.cmdletinvocationexception . dumping exceptions on heap yields following results. heap has few more exception listed below. so based on research exception happens when cmdlet encounters terminating error. method call before processrecord(). provides record-by-record processing functionality cmdlet. windows powershell runtime calls method multiple times each instance of cmdlet in pipeline. how can inspect pipeline debugger? exception object: 0000000002702940 exception type: system.management.automation.itemnotfoundexception message: cannot find path 'c:\windows\system32\windowspowershell\v1.0\modules\applicationserver\applicationserver.dll' because not exist. innerexception: <none> stacktrace (generated): sp ip function 000000001...
Read more

unity3d - Fatal error- Monodevelop-Unity failed to start -

-
Image
i installed unity monodevelop seems not been installed tried open script in unity nothing come out, saying should find software opens when have set preferences monodevelop(built-in). then reinstalled again, time, contains visual studio too. can open visual studio. when tried open monodevelop, following pop-up apear. but have installed 2 things /net , gtk. shown below. problem? 1st time-- install gtk after install unity(no monodev) 2nd time-- install monodevelop unity went monodev website , says xamarin studio not monodevelop you messed in first step. you have uninstall gtk , else installed during time. includes unity too.if possible, system restore , set restore date time before installed of this. now, install unity, make sure monodevelop selected unity , install it. don't install gtk or other software. when using unity, monodevelop must installed unity , not other website. reason because using customized version of monodevelop .
Read more