相关文章推荐
空虚的领带  ·  元素定位之css ...·  1 年前    · 
腼腆的沙发  ·  centos 6.x 安装配置 ...·  1 年前    · 
英俊的双杠  ·  Camera.Parameters ...·  1 年前    · 
精彩文章免费看

apache atlas项目搭建(1.0.0,2.0.0),centos7环境

一、前置条件

1、jdk1.8、maven3、nodejs

jdk、maven解压、配置环境变量;

nodejs安装,参考 https://www.cnblogs.com/jizhong/p/12102100.html

centos7自带python2.7.5,无需安装

2、配置环境变量 /etc/profile

JAVA_HOME=/usr/local/tools/jdk1.8.0_161
export export PATH=${JAVA_HOME}/bin:${PATH}
MAVEN_HOME=/usr/local/tools/apache-maven-3.6.3
export PATH=${MAVEN_HOME}/bin:${PATH}
NODE_HOME=/usr/local/tools/node
export PATH=${NODE_HOME}/bin:${PATH}

配置生效 source /etc/profile

3、查看版本信息:

node -v

v6.12.3

npm -v

3.10.10

python -V

Python 2.7.5

java -version

mvn –v

4、使用淘宝的npm镜像cnpm

npm install -g cnpm --registry=https://registry.npm.taobao.org

5、maven 配置文件(阿里镜像、仓库地址、jdk版本)

替换maven源,阿里云的源

打开maven配置文件 settings.xml

<mirror>
    <id>alimaven</id>
    <name>aliyun maven</name>
    <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
    <mirrorOf>central</mirrorOf>
</mirror>

指定下载资源位置

<localRepository>/usr/local/maven/repository</localRepository>

指定jdk版本(本座没有配置)

<profile>    
    <id>jdk-1.8</id>    
    <activation>    
        <activeByDefault>true</activeByDefault>    
        <jdk>1.8</jdk>    
    </activation>    
    <properties>    
        <maven.compiler.source>1.8</maven.compiler.source>    
        <maven.compiler.target>1.8</maven.compiler.target>    
        <maven.compiler.compilerVersion>1.8</maven.compiler.compilerVersion>    
    </properties>
</profile>

二、apache atlas1.0.0

1、下载源系统 http://atlas.apache.org/#/Downloads

1)、该方式编译不会内嵌HBase和Solr

mvn clean -DskipTests package -Pdist

2)、使用atlas内嵌HBase和Solr,测试用这种方式,注意:embedded-hbase-solr前面没有空格

mvn clean -DskipTests package -Pdist,embedded-hbase-solr

编译完成后的jar包位于目录:\apache-atlas-sources-1.1.0\distro\target 中,压缩包有对应的解压文件,可直接部署;或者自己加压到指定路径

3、部署,进入 apache-atlas-1.0.0 路径

/usr/local/projects/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0

用内嵌的 hbase 和 solr 启动,配置hbase和solr跟随atlas启动和停止

export MANAGE_LOCAL_HBASE=true

export MANAGE_LOCAL_SOLR=true

上面的两步已经在 atlas-env.sh 文件默认配置

bin/atlas_start.py

5、查看进程 jps

6、添加自带例子 bin/quick_start.py

Enter username for atlas :- admin Enter password for atlas :- admin

7、管理界面登录入口 http://localhost:21000 用户名:admin 密码:admin

8、停止atlas,bin/atlas_stop.py

在hbase里配置了JAVE_HOME

三、apache atlas2.0.0

编译过程和1.0.0一致,编译时间较1.0更长

编译完成后使用

/usr/local/project/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-server/apache-atlas-2.0.0

1、修改配置文件 apache-atlas-2.0.0-server/apache-atlas-2.0.0/conf/

vim atlas-env.sh
export JAVA_HOME=/usr/local/tools/jdk1.8.0_161
export MANAGE_LOCAL_HBASE=true  #如果要使用外部的zk和hbase,则改为false
export MANAGE_LOCAL_SOLR=true  #如果要是用外部的solr,则改为false
vim atlas-application.properties
# Hbase地址(对应的zk地址)配置(自带hbase会根据此端口启动一个zk实例)
atlas.graph.storage.hostname=localhost:2181 # 如果使用外部hbase,则填写外部zookeeper地址
# Solr地址配置
atlas.graph.index.search.solr.http-urls=http://localhost:8984/solr
# Kafka相关配置
atlas.notification.embedded=true # 如果要使用外部的kafka,则改为false
# 内嵌kafka会根据此端口启动一个zk实例
atlas.kafka.zookeeper.connect=localhost:9026 # 如果使用外部kafka,则填写外部zookeeper地址
atlas.kafka.bootstrap.servers=localhost:9027 # 如果使用外部kafka,则填写外部broker server地址

2、手动启动hbase

# 进入atlas自带的hbase目录
apache-atlas-2.0.0-server/apache-atlas-2.0.0/hbase 
# 启动 hbase 
bin/start-hbase.sh

3、手动启动solr,为solr创建初始化index库

# 进入atlas自带的solr目录,
apache-atlas-2.0.0-server/apache-atlas-2.0.0/solr
# 启动solr
bin/solr start -c -z localhost:2181 -p 8984 -force
# 创建初始化collections
bin/solr create -c vertex_index -shards 1 -replicationFactor 1 -force
bin/solr create -c edge_index -shards 1 -replicationFactor 1 -force
bin/solr create -c fulltext_index -shards 1 -replicationFactor 1 -force

然后打开浏览器访问solr的web服务如下,则solr启动成功 http://localhost:8984/

4、启动atlas,apache-atlas-2.0.0-server/apache-atlas-2.0.0

bin/atlas_start.py

最后添加自带例子,同1.0.0

四、访问路径

atlas:http://192.168.198.129:21000/

solr(1.0.0):http://192.168.65.139:9838/

solr(2.0.0):http://localhost:8984/

[ERROR] Java heap space -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError

没有设置 MAVEN_OPTS="-Xms2g -Xmx4g", 造成堆内存溢出

[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for org.apache.atlas:apache-atlas:1.0.0: Could not transfer artifact org.apache:apache:pom:17 from/to alimaven (http://maven.aliyun.com/nexus/content/groups/public/): Transfer failed for http://maven.aliyun.com/nexus/content/groups/public/org/apache/apache/17/apache-17.pom and 'parent.relativePath' points at wrong local POM @ line 23, column 13

编译错误:

晚上下班突然就可以了

[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-npm (install node and npm) on project atlas-dashboardv2: Could not extract the Node archive: Could not extract archive: '/usr/local/maven/repository/com/github/eirslett/node/4.4.2/node-4.4.2-linux-x64.tar.gz': EOFException -> [Help 1]
Failed to run task: 'npm install' failed
npm: command not found...

需要安装nodejs,并使用淘宝的npm镜像cnpm

npm install -g cnpm --registry=https://registry.npm.taobao.org

Creating sample types:
Exception in thread "main" org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasClientV2$API_V2@491b9b8 failed with status 503 (Service Unavailable) Response Body (<html>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 503 </title>
</head>
<h2>HTTP ERROR: 503</h2>
<p>Problem accessing /api/atlas/v2/types/typedefs/. Reason:
<pre> Service Unavailable</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
</body>
</html>
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:395)
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:323)
at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:211)
at org.apache.atlas.AtlasClientV2.createAtlasTypeDefs(AtlasClientV2.java:227)
at org.apache.atlas.examples.QuickStartV2.createTypes(QuickStartV2.java:185)
at org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:141)
at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:126)
No sample data added to Apache Atlas Server.
curl -v -u username:password http://localhost:21000/api/atlas/admin/version

solr启动失败,还是配置的问题

hbase Error: JAVA_HOME is not set

/usr/local/project/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-bin/apache-atlas-2.0.0/hbase/conf

atlas 1.0.0 配置了JAVA_HOME

hbase-env.sh

cannot create temp file for here-document: No space left on device

空间满了 df -h

Caused by: org.apache.solr.common.SolrException: Cannot connect to cluster at localhost:2181: cluster not found/not ready
at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:385)
at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:141)
at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:383)
at org.janusgraph.diskstorage.solr.Solr6Index.<init>(Solr6Index.java:218)

2181 zookeeper 默认端口

还是配置的问题,atlas2.0.0

最后编辑于:2020-06-17 18:11