Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
D
DataX
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
risk-feature
DataX
Commits
7b0216f7
Unverified
Commit
7b0216f7
authored
May 23, 2019
by
binaryWorld
Committed by
GitHub
May 23, 2019
Browse files
Options
Browse Files
Download
Plain Diff
Merge pull request #343 from bakea/master
fix#342 support phoenix table name lower case.
parents
6434d8a2
07cb166c
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
7 additions
and
6 deletions
+7
-6
README.md
README.md
+1
-0
HBase20SQLReaderHelper.java
...ugin/reader/hbase20xsqlreader/HBase20SQLReaderHelper.java
+3
-3
HBase20xSQLWriterTask.java
...lugin/writer/hbase20xsqlwriter/HBase20xSQLWriterTask.java
+3
-3
No files found.
README.md
View file @
7b0216f7
...
@@ -48,6 +48,7 @@ DataX目前已经有了比较全面的插件体系,主流的RDBMS数据库、N
...
@@ -48,6 +48,7 @@ DataX目前已经有了比较全面的插件体系,主流的RDBMS数据库、N
| | Hbase0.94 | √ | √ |
[
读
](
https://github.com/alibaba/DataX/blob/master/hbase094xreader/doc/hbase094xreader.md
)
、
[
写
](
https://github.com/alibaba/DataX/blob/master/hbase094xwriter/doc/hbase094xwriter.md
)
|
| | Hbase0.94 | √ | √ |
[
读
](
https://github.com/alibaba/DataX/blob/master/hbase094xreader/doc/hbase094xreader.md
)
、
[
写
](
https://github.com/alibaba/DataX/blob/master/hbase094xwriter/doc/hbase094xwriter.md
)
|
| | Hbase1.1 | √ | √ |
[
读
](
https://github.com/alibaba/DataX/blob/master/hbase11xreader/doc/hbase11xreader.md
)
、
[
写
](
https://github.com/alibaba/DataX/blob/master/hbase11xwriter/doc/hbase11xwriter.md
)
|
| | Hbase1.1 | √ | √ |
[
读
](
https://github.com/alibaba/DataX/blob/master/hbase11xreader/doc/hbase11xreader.md
)
、
[
写
](
https://github.com/alibaba/DataX/blob/master/hbase11xwriter/doc/hbase11xwriter.md
)
|
| | Phoenix4.x | √ | √ |
[
读
](
https://github.com/alibaba/DataX/blob/master/hbase11xsqlreader/doc/hbase11xsqlreader.md
)
、
[
写
](
https://github.com/alibaba/DataX/blob/master/hbase11xsqlwriter/doc/hbase11xsqlwriter.md
)
|
| | Phoenix4.x | √ | √ |
[
读
](
https://github.com/alibaba/DataX/blob/master/hbase11xsqlreader/doc/hbase11xsqlreader.md
)
、
[
写
](
https://github.com/alibaba/DataX/blob/master/hbase11xsqlwriter/doc/hbase11xsqlwriter.md
)
|
| | Phoenix5.x | √ | √ |
[
读
](
https://github.com/alibaba/DataX/blob/master/hbase20xsqlreader/doc/hbase20xsqlreader.md
)
、
[
写
](
https://github.com/alibaba/DataX/blob/master/hbase20xsqlwriter/doc/hbase20xsqlwriter.md
)
|
| | MongoDB | √ | √ |
[
读
](
https://github.com/alibaba/DataX/blob/master/mongoreader/doc/mongoreader.md
)
、
[
写
](
https://github.com/alibaba/DataX/blob/master/mongowriter/doc/mongowriter.md
)
|
| | MongoDB | √ | √ |
[
读
](
https://github.com/alibaba/DataX/blob/master/mongoreader/doc/mongoreader.md
)
、
[
写
](
https://github.com/alibaba/DataX/blob/master/mongowriter/doc/mongowriter.md
)
|
| | Hive | √ | √ |
[
读
](
https://github.com/alibaba/DataX/blob/master/hdfsreader/doc/hdfsreader.md
)
、
[
写
](
https://github.com/alibaba/DataX/blob/master/hdfswriter/doc/hdfswriter.md
)
|
| | Hive | √ | √ |
[
读
](
https://github.com/alibaba/DataX/blob/master/hdfsreader/doc/hdfsreader.md
)
、
[
写
](
https://github.com/alibaba/DataX/blob/master/hdfswriter/doc/hdfswriter.md
)
|
| 无结构化数据存储 | TxtFile | √ | √ |
[
读
](
https://github.com/alibaba/DataX/blob/master/txtfilereader/doc/txtfilereader.md
)
、
[
写
](
https://github.com/alibaba/DataX/blob/master/txtfilewriter/doc/txtfilewriter.md
)
|
| 无结构化数据存储 | TxtFile | √ | √ |
[
读
](
https://github.com/alibaba/DataX/blob/master/txtfilereader/doc/txtfilereader.md
)
、
[
写
](
https://github.com/alibaba/DataX/blob/master/txtfilewriter/doc/txtfilewriter.md
)
|
...
...
hbase20xsqlreader/src/main/java/com/alibaba/datax/plugin/reader/hbase20xsqlreader/HBase20SQLReaderHelper.java
View file @
7b0216f7
...
@@ -49,9 +49,9 @@ public class HBase20SQLReaderHelper {
...
@@ -49,9 +49,9 @@ public class HBase20SQLReaderHelper {
String
schema
=
configuration
.
getString
(
Key
.
SCHEMA
,
null
);
String
schema
=
configuration
.
getString
(
Key
.
SCHEMA
,
null
);
String
tableName
=
configuration
.
getNecessaryValue
(
Key
.
TABLE
,
HBase20xSQLReaderErrorCode
.
REQUIRED_VALUE
);
String
tableName
=
configuration
.
getNecessaryValue
(
Key
.
TABLE
,
HBase20xSQLReaderErrorCode
.
REQUIRED_VALUE
);
if
(
schema
!=
null
&&
!
schema
.
isEmpty
())
{
if
(
schema
!=
null
&&
!
schema
.
isEmpty
())
{
fullTableName
=
schema
+
"."
+
tableName
;
fullTableName
=
"\""
+
schema
+
"\".\""
+
tableName
+
"\""
;
}
else
{
}
else
{
fullTableName
=
tableName
;
fullTableName
=
"\""
+
tableName
+
"\""
;
}
}
// 如果列名未配置,默认读取全部列*
// 如果列名未配置,默认读取全部列*
columnNames
=
configuration
.
getList
(
Key
.
COLUMN
,
String
.
class
);
columnNames
=
configuration
.
getList
(
Key
.
COLUMN
,
String
.
class
);
...
@@ -248,7 +248,7 @@ public class HBase20SQLReaderHelper {
...
@@ -248,7 +248,7 @@ public class HBase20SQLReaderHelper {
String
querySql
;
String
querySql
;
StringBuilder
columnBuilder
=
new
StringBuilder
();
StringBuilder
columnBuilder
=
new
StringBuilder
();
for
(
String
columnName
:
columnNames
)
{
for
(
String
columnName
:
columnNames
)
{
columnBuilder
.
append
(
columnName
).
append
(
","
);
columnBuilder
.
append
(
"\""
).
append
(
columnName
).
append
(
"\
","
);
}
}
columnBuilder
.
setLength
(
columnBuilder
.
length
()
-
1
);
columnBuilder
.
setLength
(
columnBuilder
.
length
()
-
1
);
if
(
StringUtils
.
isBlank
(
where
))
{
if
(
StringUtils
.
isBlank
(
where
))
{
...
...
hbase20xsqlwriter/src/main/java/com/alibaba/datax/plugin/writer/hbase20xsqlwriter/HBase20xSQLWriterTask.java
View file @
7b0216f7
...
@@ -72,9 +72,9 @@ public class HBase20xSQLWriterTask {
...
@@ -72,9 +72,9 @@ public class HBase20xSQLWriterTask {
batchSize
=
configuration
.
getInt
(
Key
.
BATCHSIZE
,
Constant
.
DEFAULT_BATCH_ROW_COUNT
);
batchSize
=
configuration
.
getInt
(
Key
.
BATCHSIZE
,
Constant
.
DEFAULT_BATCH_ROW_COUNT
);
String
schema
=
configuration
.
getString
(
Key
.
SCHEMA
);
String
schema
=
configuration
.
getString
(
Key
.
SCHEMA
);
String
tableName
=
configuration
.
getNecessaryValue
(
Key
.
TABLE
,
HBase20xSQLWriterErrorCode
.
REQUIRED_VALUE
);
String
tableName
=
configuration
.
getNecessaryValue
(
Key
.
TABLE
,
HBase20xSQLWriterErrorCode
.
REQUIRED_VALUE
);
fullTableName
=
tableName
;
fullTableName
=
"\""
+
tableName
+
"\""
;
if
(
schema
!=
null
&&
!
schema
.
isEmpty
())
{
if
(
schema
!=
null
&&
!
schema
.
isEmpty
())
{
fullTableName
=
schema
+
"."
+
tableName
;
fullTableName
=
"\""
+
schema
+
"\".\""
+
tableName
+
"\""
;
}
}
columns
=
configuration
.
getList
(
Key
.
COLUMN
,
String
.
class
);
columns
=
configuration
.
getList
(
Key
.
COLUMN
,
String
.
class
);
if
(
pstmt
==
null
)
{
if
(
pstmt
==
null
)
{
...
@@ -125,7 +125,7 @@ public class HBase20xSQLWriterTask {
...
@@ -125,7 +125,7 @@ public class HBase20xSQLWriterTask {
int
[]
types
=
new
int
[
numberOfColumnsToWrite
];
int
[]
types
=
new
int
[
numberOfColumnsToWrite
];
StringBuilder
columnNamesBuilder
=
new
StringBuilder
();
StringBuilder
columnNamesBuilder
=
new
StringBuilder
();
for
(
String
columnName
:
columns
)
{
for
(
String
columnName
:
columns
)
{
columnNamesBuilder
.
append
(
columnName
).
append
(
","
);
columnNamesBuilder
.
append
(
"\""
).
append
(
columnName
).
append
(
"\
","
);
}
}
columnNamesBuilder
.
setLength
(
columnNamesBuilder
.
length
()
-
1
);
columnNamesBuilder
.
setLength
(
columnNamesBuilder
.
length
()
-
1
);
// 查询一条数据获取表meta信息
// 查询一条数据获取表meta信息
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment