BigTable - local instance

Is there a way to specify the host/port of the BigTable instance? We'd like to use https://github.com/spotify/docker-bigtable for local testing.

@callmehiphop

@callmehiphop we can probably do this the same way we do with Datastore, with an 'apiEndpoint' option. Want to tack it onto your v2 PR?

Sure!

On second thought I think we should do this post-merge - this feature would have to wait for v2 completion anyways and it would ship around the same time regardless of whether or not we lump them together.

@lesv

@lesv is it possible to connect to a Bigtable emulator host through gRPC?

I tried to make a request, and the emulator prints:

[bigtable] 2016/09/02 12:28:41 transport: http2Server.HandleStreams received bogus greeting from client: "\x16\x03\x01\x00\xa6\x01\x00\x00\xa2\x03\x03\x9bp|Mb)z\xf7\xe8\x17\xba3\xbe"
[bigtable] 2016/09/02 12:28:46 transport: http2Server.HandleStreams failed to receive the preface from client: EOF

And output from the script that made the request:

E0902 12:28:41.194034000 123145321807872 ssl_transport_security.c:945] Handshake failed with fatal error SSL_ERROR_SSL: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number.
E0902 12:28:41.194212000 123145321807872 handshake.c:241] Handshake failed with error TSI_PROTOCOL_FAILURE
E0902 12:28:41.194218000 123145321807872 secure_channel_create.c:99] Secure handshake failed with error 1.
E0902 12:28:46.549982000 123145338609664 tcp_client_posix.c:191] failed to connect to 'ipv4:127.0.0.1:8051': timeout occurred

@stephenplusplus

@stephenplusplus The emulator only supports plaintext negotiation, try to disable TLS. Maybe this is grpc.ChannelCredentials.createInsecure()?

Thank you, I forgot about that! I've run into the next error which is returned from gRPC:

Error: unknown service google.bigtable.admin.v2.BigtableInstanceAdmin
      at Function.GrpcService.decorateError_ (node_modules/@google-cloud/common/src/grpc-service.js:494:34)
      at Object.callback (node_modules/@google-cloud/common/src/grpc-service.js:253:35)
      at node_modules/grpc/src/node/src/client.js:420:14

This is trying to create an instance-- are there any methods/operations/services that aren't accessible through the emulator?

It doesn't support any instance-related services. So just use any project/instance and start by creating a table.

Hi, I finally got around to giving this a shot and seems to be hanging on this RPC:

const client = bigtable({
  projectId: config.get('gcloud.projectId'),
  zone: config.get('bigtable.zone'),
  maxRetries: 6,
  credentials,
}).instance(config.get('bigtable.instance'));

...

  async init() {
    this.table = await new Promise((resolve, reject) => {
      client.table('events').get({
        autoCreate: true,
        families: [{
          name: 'event',
          rule: {
            versions: 1,
          },
        }],
      }, (err, table) => {
        if (err) return void reject(err);
        resolve(table);
      });
    });

I'm not seeing any logging in the bigtable process:

➜  ~ gcloud beta emulators bigtable start --host-port localhost:8721 --quiet
Executing: /Users/mata/Downloads/google-cloud-sdk/platform/bigtable-emulator/cbtemulator --host=localhost --port=8721
[bigtable] Cloud Bigtable emulator running on 127.0.0.1:8721

@garye

Looks like there's a discrepancy between the emulator and the upstream API. When autoCreate is set to true we first attempt to get the object in question and in the event of a 404 we'll attempt to create said object. The emulator appears to be returning a 500 error instead of a 404.

@garye do we know who's in charge of the emulator?

@arbesfeld In the meantime you could try something like

async init() {
  const table = client.table('events');

  this.table = await new Promise((resolve, reject) => {
    table.get(err => {
      if (!err) return resolve(table);

      table.create({ families: [...] }, err => {
        if (err) return void reject(err);
        resolve(table);
      });
    });
  });
}

@callmehiphop

@callmehiphop I am :) The emulator is actually just this:
https://github.com/GoogleCloudPlatform/google-cloud-go/blob/master/bigtable/bttest/inmem.go

Let me look into changing the error code. So the autoCreate option calls GetTable?

and grpc NotFound gets treated as a 404, right?

So the autoCreate option calls GetTable?

Correct! First we call GetTable and if that returns a 404 (grpc 5) then we call CreateTable

and grpc NotFound gets treated as a 404, right?

Yep! Sorry I didn't mention that up front, we map grpc codes to HTTP codes to make it easier for the user.

There's a fix in for this: googleapis/google-cloud-go@1d0367c

Will take at least a week to make its way into the gcloud emulator. Let me know if you're up for trying against the go app itself.

@callmehiphop

@callmehiphop @arbesfeld Any issues left connecting to the emulator or are things looking good here?

It works! Thank you

build: correct repository for publication failures (#1506)

chore(deps): update dependency google-auth-library to v10.2.0 (#1506)

feat: add `gcf-owl-bot[bot]` to `ignoreAuthors` (#1506)

feat: customize retry behavior implementation (#1474) (#1493)

chore: [node] add auto-approve templates, and install dependencies wi…

feat: customize retry behavior implementation (googleapis#1474) (goog…

Navigation Menu

Repository navigation

BigTable - local instance #1506

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

BigTable - local instance #1506

Description

Activity

stephenplusplus commented on Aug 18, 2016

callmehiphop commented on Aug 18, 2016

callmehiphop commented on Aug 21, 2016

stephenplusplus commented on Sep 2, 2016

garye commented on Sep 2, 2016

stephenplusplus commented on Sep 2, 2016

garye commented on Sep 2, 2016

arbesfeld commented on Oct 19, 2016

callmehiphop commented on Oct 20, 2016

garye commented on Oct 21, 2016

garye commented on Oct 21, 2016

callmehiphop commented on Oct 21, 2016

garye commented on Oct 25, 2016

garye commented on Nov 18, 2016

arbesfeld commented on Nov 18, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions