apache atlas项目搭建(1.0.0,2.0.0),centos7环境
一、前置条件
1、jdk1.8、maven3、nodejs
jdk、maven解压、配置环境变量;
nodejs安装,参考 https://www.cnblogs.com/jizhong/p/12102100.html
centos7自带python2.7.5,无需安装
2、配置环境变量 /etc/profile
JAVA_HOME=/usr/local/tools/jdk1.8.0_161
export export PATH=${JAVA_HOME}/bin:${PATH}
MAVEN_HOME=/usr/local/tools/apache-maven-3.6.3
export PATH=${MAVEN_HOME}/bin:${PATH}
NODE_HOME=/usr/local/tools/node
export PATH=${NODE_HOME}/bin:${PATH}
配置生效 source /etc/profile
3、查看版本信息:
node -v
v6.12.3
npm -v
3.10.10
python -V
Python 2.7.5
java -version
mvn –v
4、使用淘宝的npm镜像cnpm
npm install -g cnpm --registry=https://registry.npm.taobao.org
5、maven 配置文件(阿里镜像、仓库地址、jdk版本)
替换maven源,阿里云的源
打开maven配置文件 settings.xml
<mirror>
<id>alimaven</id>
<name>aliyun maven</name>
<url>http://maven.aliyun.com/nexus/content/groups/public/</url>
<mirrorOf>central</mirrorOf>
</mirror>
指定下载资源位置
<localRepository>/usr/local/maven/repository</localRepository>
指定jdk版本(本座没有配置)
<profile>
<id>jdk-1.8</id>
<activation>
<activeByDefault>true</activeByDefault>
<jdk>1.8</jdk>
</activation>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<maven.compiler.compilerVersion>1.8</maven.compiler.compilerVersion>
</properties>
</profile>
二、apache atlas1.0.0
1、下载源系统 http://atlas.apache.org/#/Downloads
1)、该方式编译不会内嵌HBase和Solr
mvn clean -DskipTests package -Pdist
2)、使用atlas内嵌HBase和Solr,测试用这种方式,注意:embedded-hbase-solr前面没有空格
mvn clean -DskipTests package -Pdist,embedded-hbase-solr
编译完成后的jar包位于目录:\apache-atlas-sources-1.1.0\distro\target 中,压缩包有对应的解压文件,可直接部署;或者自己加压到指定路径
3、部署,进入 apache-atlas-1.0.0 路径
/usr/local/projects/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0
用内嵌的 hbase 和 solr 启动,配置hbase和solr跟随atlas启动和停止
export MANAGE_LOCAL_HBASE=true
export MANAGE_LOCAL_SOLR=true
上面的两步已经在 atlas-env.sh 文件默认配置
bin/atlas_start.py
5、查看进程 jps
6、添加自带例子 bin/quick_start.py
Enter username for atlas :- admin Enter password for atlas :- admin
7、管理界面登录入口 http://localhost:21000 用户名:admin 密码:admin
8、停止atlas,bin/atlas_stop.py
在hbase里配置了JAVE_HOME
三、apache atlas2.0.0
编译过程和1.0.0一致,编译时间较1.0更长
编译完成后使用
/usr/local/project/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-server/apache-atlas-2.0.0
1、修改配置文件 apache-atlas-2.0.0-server/apache-atlas-2.0.0/conf/
vim atlas-env.sh
export JAVA_HOME=/usr/local/tools/jdk1.8.0_161
export MANAGE_LOCAL_HBASE=true #如果要使用外部的zk和hbase,则改为false
export MANAGE_LOCAL_SOLR=true #如果要是用外部的solr,则改为false
vim atlas-application.properties
# Hbase地址(对应的zk地址)配置(自带hbase会根据此端口启动一个zk实例)
atlas.graph.storage.hostname=localhost:2181 # 如果使用外部hbase,则填写外部zookeeper地址
# Solr地址配置
atlas.graph.index.search.solr.http-urls=http://localhost:8984/solr
# Kafka相关配置
atlas.notification.embedded=true # 如果要使用外部的kafka,则改为false
# 内嵌kafka会根据此端口启动一个zk实例
atlas.kafka.zookeeper.connect=localhost:9026 # 如果使用外部kafka,则填写外部zookeeper地址
atlas.kafka.bootstrap.servers=localhost:9027 # 如果使用外部kafka,则填写外部broker server地址
2、手动启动hbase
# 进入atlas自带的hbase目录
apache-atlas-2.0.0-server/apache-atlas-2.0.0/hbase
# 启动 hbase
bin/start-hbase.sh
3、手动启动solr,为solr创建初始化index库
# 进入atlas自带的solr目录,
apache-atlas-2.0.0-server/apache-atlas-2.0.0/solr
# 启动solr
bin/solr start -c -z localhost:2181 -p 8984 -force
# 创建初始化collections
bin/solr create -c vertex_index -shards 1 -replicationFactor 1 -force
bin/solr create -c edge_index -shards 1 -replicationFactor 1 -force
bin/solr create -c fulltext_index -shards 1 -replicationFactor 1 -force
然后打开浏览器访问solr的web服务如下,则solr启动成功 http://localhost:8984/
4、启动atlas,apache-atlas-2.0.0-server/apache-atlas-2.0.0
bin/atlas_start.py
最后添加自带例子,同1.0.0
四、访问路径
atlas:http://192.168.198.129:21000/
solr(1.0.0):http://192.168.65.139:9838/
solr(2.0.0):http://localhost:8984/
[ERROR] Java heap space -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError
没有设置 MAVEN_OPTS="-Xms2g -Xmx4g", 造成堆内存溢出
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for org.apache.atlas:apache-atlas:1.0.0: Could not transfer artifact org.apache:apache:pom:17 from/to alimaven (http://maven.aliyun.com/nexus/content/groups/public/): Transfer failed for http://maven.aliyun.com/nexus/content/groups/public/org/apache/apache/17/apache-17.pom and 'parent.relativePath' points at wrong local POM @ line 23, column 13
编译错误:
晚上下班突然就可以了
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-npm (install node and npm) on project atlas-dashboardv2: Could not extract the Node archive: Could not extract archive: '/usr/local/maven/repository/com/github/eirslett/node/4.4.2/node-4.4.2-linux-x64.tar.gz': EOFException -> [Help 1]
Failed to run task: 'npm install' failed
npm: command not found...
需要安装nodejs,并使用淘宝的npm镜像cnpm
npm install -g cnpm --registry=https://registry.npm.taobao.org
Creating sample types:
Exception in thread "main" org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasClientV2$API_V2@491b9b8 failed with status 503 (Service Unavailable) Response Body (<html>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 503 </title>
</head>
<h2>HTTP ERROR: 503</h2>
<p>Problem accessing /api/atlas/v2/types/typedefs/. Reason:
<pre> Service Unavailable</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
</body>
</html>
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:395)
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:323)
at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:211)
at org.apache.atlas.AtlasClientV2.createAtlasTypeDefs(AtlasClientV2.java:227)
at org.apache.atlas.examples.QuickStartV2.createTypes(QuickStartV2.java:185)
at org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:141)
at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:126)
No sample data added to Apache Atlas Server.
curl -v -u username:password http://localhost:21000/api/atlas/admin/version
solr启动失败,还是配置的问题
hbase Error: JAVA_HOME is not set
/usr/local/project/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-bin/apache-atlas-2.0.0/hbase/conf
atlas 1.0.0 配置了JAVA_HOME
hbase-env.sh
cannot create temp file for here-document: No space left on device
空间满了 df -h
Caused by: org.apache.solr.common.SolrException: Cannot connect to cluster at localhost:2181: cluster not found/not ready
at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:385)
at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:141)
at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:383)
at org.janusgraph.diskstorage.solr.Solr6Index.<init>(Solr6Index.java:218)
2181 zookeeper 默认端口
还是配置的问题,atlas2.0.0