Hive - Thrift - readMessageBegin中缺少版本,旧客户端

Hive - Thrift - Missing version in readMessageBegin, old client?

本文关键字:版本 客户端 Thrift readMessageBegin Hive      更新时间:2023-09-26

您好,我正在尝试构建一个 nodejs 客户端,用于使用 thrift 查询我的 hive 数据库,但我遇到了一个奇怪的问题......我已经用节俭生成了我的nodejs客户端API(thrift -r --gen js:node TCLIService.thrift TCLIService是定义Hive服务的节俭文件),现在我尝试连接到Hive,但我的OpenSession正在等待中...也许我没有做正确的电话,但我在网上没有找到任何最近的内容(每个节俭/节点/蜂巢项目都有 4 或 5 年的历史)。你能看看并告诉我我是否做错了吗?谢谢

TCLIService.thrift :

// OpenSession()
//
// Open a session (connection) on the server against
// which operations may be executed.
struct TOpenSessionReq {
    // The version of the HiveServer2 protocol that the client is using.
    1: required TProtocolVersion client_protocol = TProtocolVersion.HIVE_CLI_SERVICE_PROTOCOL_V8
    // Username and password for authentication.
    // Depending on the authentication scheme being used,
    // this information may instead be provided by a lower
    // protocol layer, in which case these fields may be
    // left unset.
    2: optional string username
    3: optional string password
    // Configuration overlay which is applied when the session is
    // first created.
    4: optional map<string, string> configuration
}
service TCLIService {
    TOpenSessionResp OpenSession(1:TOpenSessionReq req);
}

Nodejs 代码: var sessionHandle = '';

function openSession(config) {
    openSessReq = new ttypes.TOpenSessionReq();
    openSessReq.client_protocol = ttypes.TProtocolVersion.HIVE_CLI_SERVICE_PROTOCOL_V8;
    openSessReq.username = config.hiveUser;
    openSessReq.password = config.hivePassword;
    console.log("openSessReq = " + JSON.stringify(openSessReq));
    console.log("Before OpenSession : sessionHandle = " + sessionHandle);
    sessionHandle = client.OpenSession(openSessReq, function (err, result){
        console.log('handler fired ... ');
        if (err) {
            console.error("error : " + err);
        } else {
            console.log("result : " + result);
        }
    });
    console.log("After OpenSession : sessionHandle = " + sessionHandle);
}
connection.on('error', function(err) {
    /*Error detected, print the error*/
    console.error(err);
    /*Close the program with error code*/
    process.exit(1)
});
console.log('Error handler registered ...')
connection.on('connect', function(){
    console.log('Connected ...');
    /*Init session*/
    console.log(client);
    openSession(config);
}

我的输出如下:

CreateConnection DONE ...
Error handler registered ...
Connect handler registered ...
Connected ...
{ output:
   TBufferedTransport {
     defaultReadBufferSize: 1024,
     writeBufferSize: 512,
     inBuf: <Buffer e8 19 2b 03 00 00 00 00 b0 1a 2b 03 00 00 00 00 e8 18 2b 03 00 00 00 00 f0 18 2b 03 00 00 00 00 b0 19 2b 03 00 00 00 00 78 1a 2b 03 00 00 00 00 70 1d ... >,
     readCursor: 0,
     writeCursor: 0,
     outBuffers: [],
     outCount: 0,
     onFlush: [Function],
     client: [Circular] },
  pClass: [Function: TBinaryProtocol],
  _seqid: 0,
  _reqs: {} }
openSessReq = {"client_protocol":7,"username":"root","password":"root","configuration":null}
Before OpenSession : sessionHandle =
After OpenSession : sessionHandle = undefined
^C

脚本无限期运行...如果我更改端口或关闭 HiveServer,我会遇到错误,因此连接正常工作,但我无法找出为什么 openSession 不是!感谢您的帮助

编辑:我已经检查了我的日志,NOSASL 识别是问题所在......我已经修复了(我认为)这个问题,现在收到以下错误:

客户:

{ [Error: write EPIPE]
 code: 'EPIPE',
 errno: 'EPIPE',
 syscall: 'write',
 address: undefined }

服务器:

2016-02-17 13:36:37,152 ERROR org.apache.thrift.server.TThreadPoolServer: [HiveServer2-Handler-Pool: Thread-24]: Thrift error occurred during processing of message.
    org.apache.thrift.protocol.TProtocolException: Missing version in readMessageBegin, old client?
        at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:228)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
        at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

我已经读到这个问题必须通过使用错误的协议层来提出......我正在使用TBinary协议有问题吗?

我已经通过执行以下操作解决了我的问题:

使用 beewax 检查我的 HiveServer2 的协议版本,并查询我的 HiveServer2(

使用 Hue)并检查 HiveServer2 日志(我发现它是 Hive 0.15.0 HIVE_CLI_SERVICE_PROTOCOL_V7的)

通过修改配置站点在我的 HiveServer2 上启用 NOSASL 身份验证.xml

尝试检查环境的三个部分,默认情况下,这三个部分都在同一个配置文件( hive-site.xml ) 中:

  1. 传输模式选中hive.server2.transport.mode,您必须在服务和客户端中使用相同的模式。

  2. SSL 启用选中hive.server2.use.SSL,您可以尝试关闭以确保与此配置无关。

  3. 认证检查hive.server2.authentication,因为有几种(NONE,NOSASL,KERBEROS,LDAP,CUSTOM)的身份验证模式,请确保使用正确的一种。

附言。您可以尝试通过控制台连接到hs2,看看服务器是否有任何问题。