Apache Carbondata on Preto

北斗云 2019-08-03

hadoop Apache Maven node spark Server facebook plugin presto carbondata

1.download

apache carbondata - 1.5.3
apache carbondata - 1.5.4
apache spark - 2.3.2
apache hadoop - 2.7.2
presto server - 0.210
presto client - 0.210

2.Configurations

mkdir -p etc/catalog -- in presto server
touch carbondata.properties -- in presto server /conf

a).vi config.properties

######## Coordinator Configurations ##########
#
#coordinator=true
#node-scheduler.include-coordinator=false
#http-server.http.port=8086
#query.max-memory=5GB
#query.max-total-memory-per-node=5GB
#query.max-memory-per-node=3GB
#memory.heap-headroom-per-node=1GB
#discovery-server.enabled=true
#discovery.uri=http://hostname:8086
#task.max-worker-threads=4
#optimizer.dictionary-aggregation=true
#optimizer.optimize-hash-generation = false
#
##############################################
########## Worker Configurations####
#
#coordinator=false
#http-server.http.port=8086
#query.max-memory=5GB
#query.max-memory-per-node=2GB
#discovery.uri=<coordinator_ip>:8086
#
####################################

b).vi jvm.config

-server
-Xmx16G
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+UseGCOverheadLimit
-XX:+ExplicitGCInvokesConcurrent
-XX:+HeapDumpOnOutOfMemoryError
-XX:OnOutOfMemoryError=kill -9 %p

c).vi log.properties

com.facebook.presto=INFO

d).vi node.properties

node.environment=production
node.id=c5a9ab1a-a9fe-11e9-96d8-000c2909bcd1
node.data-dir=/home/carbondata/presto-server-0.210/data

3.plugin

mkdir -p plugin/carbondata -- in presto server

cp apache-carbondata-1.5.3-bin-spark2.3.2-hadoop2.7.2.jar /home/carbondata/presto-server-0.210/plugin/carbondata

4.start server

./presto-server-0.210/bin/launcher start

./presto-server-0.210/bin/launcher run

5.presto-cli

mv presto-cli-0.210-executable.jar presto

./presto
登录 后评论
下一篇
冒顿单于
6662人浏览
2019-08-28
相关推荐
提交PR的git 流程
1930人浏览
2017-03-31 21:34:50
CarbonData集群模式体验
4158人浏览
2016-09-11 18:36:43
让CarbonData使用更简单
3596人浏览
2017-03-31 21:24:01
Apache Zookeeper常用命令
2220人浏览
2019-08-22 17:00:07
Alluxio使用——TensorFlow篇
441人浏览
2019-08-29 13:29:53
Apache AirFlow安装部署
219人浏览
2019-09-05 15:40:19
Apache AirFlow开篇
47人浏览
2019-09-05 18:21:26
0
1
0
244