Python通过kerberos安全认证操作kafka方式
如何通过kerberos认证.
1.安装kerberos客户端
centos:
yum install krb5-workstation
使用which kinit查看是否安装成功;
2.拷贝kerberos配置文件
conf目录下krb5.conf和kafka.keytab和jaas.conf拷贝到客户端机器的etc目录, 同时,krb5.conf中的kdc集群主机名和ip配置到客户端机器hosts配置文件中
3.kinit客户端通过kerberos认证
获取principal
klist -kt kafka.keytab
4.安装python-gssapi
pip install gssapi
遇到的问题,如下:
a.在linux中执行wget命令提示 -bash: wget: command not found 解决方法
yum -y install wget
b.报错:bash: pip: command not found
wget https://bootstrap.pypa.io/get-pip.py python get-pip.py pip -v #查看pip版本
python -m pip install --upgrade --force pip easy_install -u setuptools pip install --upgrade setuptools
3.pip安装出现command “python setup.py egg_info” failed with error code 1 的解决方法
traceback (most recent call last): file “”, line 1, in file “/tmp/pip-install-6hfde3/gssapi/setup.py”, line 109, in raise exception("could not find main gssapi shared library. please " exception: could not find main gssapi shared library. please try setting gssapi_main_lib yourself or setting enable_support_detection to ‘false'
command “python setup.py egg_info” failed with error code 1 in /tmp/pip-install-6hfde3/gssapi/
yum install -y krb5-devel.x86_64
4.关于error: command ‘gcc' failed with exit status 1错误的解决方法
yum install gcc python-devel
安装kafka-python
pip install kafka-python
初始化环境变量
export kafka_opts="-djava.security.auth.login.config=/etc/conf/jaas.conf -djava.security.krb5.conf=/etc/krb5.conf"
python操作kafka样例
from kafka import kafkaproducer from kafka.errors import kafkaerror import os class kafka_producer(): def __init__(self, kafkahost, kafkaport, kafkatopic): self.kafkahost = kafkahost self.kafkaport = kafkaport self.kafkatopic = kafkatopic self.producer = kafkaproducer( bootstrap_servers = '{kafka_host}:{kafka_port}'.format(kafka_host=self.kafkahost,kafka_port=self.kafkaport), security_protocol="sasl_plaintext", sasl_mechanism="gssapi", sasl_kerberos_service_name="kafka", compression_type='gzip' #压缩方式 ) def sendfiledata(self, params): try: f = open(params,'rb') parmasmessage = f.read(-1).strip() producer = self.producer producer.send(self.kafkatopic, parmasmessage) producer.flush() except kafkaerror as e: print (e) def main(): filepath = "/home/public/data/" topic = "demo" producer = kafka_producer("xxx.xx.xx.xx","9092",topic) dirlist = os.listdir(filepath) for filename in dirlist: producer.sendfiledata(filepath+filename) print('send success!!!') if __name__=='__main__': main()
以上这篇python通过kerberos安全认证操作kafka方式就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持。
上一篇: python访问hdfs的操作
下一篇: Java之路---Day06