Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Q
qg-rt-dc
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
data-spider
qg-rt-dc
Commits
10ed45e2
Commit
10ed45e2
authored
Jun 28, 2019
by
data爬虫-冯 军凯
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
设备指纹实时数据清洗增加传入数据源类型
parent
9ac7a242
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
10 additions
and
10 deletions
+10
-10
DeviceFingerInfoService.java
.../qgdataservice/service/hbase/DeviceFingerInfoService.java
+2
-2
DeviceFingerInfoCleaningService.java
...service/service/tidb/DeviceFingerInfoCleaningService.java
+4
-4
JdbcExecuters.java
...java/cn/quantgroup/qgdataservice/utils/JdbcExecuters.java
+4
-4
No files found.
qg-data-service/src/main/java/cn/quantgroup/qgdataservice/service/hbase/DeviceFingerInfoService.java
View file @
10ed45e2
...
...
@@ -12,11 +12,11 @@ import com.lkb.data.hbase.row.MobileFingerPrintDataRow;
*/
public
class
DeviceFingerInfoService
{
public
static
void
storageDeviceFingerInfo
(
MobileFingerPrintDataRow
mobileFingerPrintDataRow
,
String
uuid
,
boolean
ka
)
{
public
static
void
storageDeviceFingerInfo
(
MobileFingerPrintDataRow
mobileFingerPrintDataRow
,
String
uuid
,
boolean
ka
,
String
type
)
{
MobileFingerPrintDataService
.
put
(
mobileFingerPrintDataRow
);
DeviceFingerInfoCleaningService
.
cleanningAndSaveDeviceFingersInfo
(
mobileFingerPrintDataRow
,
uuid
,
ka
);
DeviceFingerInfoCleaningService
.
cleanningAndSaveDeviceFingersInfo
(
mobileFingerPrintDataRow
,
uuid
,
ka
,
type
);
}
...
...
qg-data-service/src/main/java/cn/quantgroup/qgdataservice/service/tidb/DeviceFingerInfoCleaningService.java
View file @
10ed45e2
...
...
@@ -178,7 +178,7 @@ public class DeviceFingerInfoCleaningService {
longColumns
.
add
(
"systemUpTime"
);
}
public
static
void
cleanningAndSaveDeviceFingersInfo
(
MobileFingerPrintDataRow
mobileFingerPrintDataRow
,
String
uuid
,
boolean
ka
)
{
public
static
void
cleanningAndSaveDeviceFingersInfo
(
MobileFingerPrintDataRow
mobileFingerPrintDataRow
,
String
uuid
,
boolean
ka
,
String
type
)
{
ThreadPoolExecutorUtils
.
getThreadPoolUtil
().
execute
(
new
Runnable
()
{
@Override
...
...
@@ -228,12 +228,12 @@ public class DeviceFingerInfoCleaningService {
paramList
.
add
(
list
);
}
catch
(
Exception
e
)
{
log
.
error
(
"设备指纹清洗异常, uuid: {} , ka: {} ,
param: {} "
,
uuid
,
ka
,
JSON
.
toJSONString
(
mobileFingerPrintDataRow
));
log
.
error
(
"设备指纹清洗异常, uuid: {} , ka: {} ,
type: {} , param: {} "
,
uuid
,
ka
,
type
,
JSON
.
toJSONString
(
mobileFingerPrintDataRow
));
}
JdbcExecuters
.
prepareBatchUpdate
(
Constant
.
SQL
.
DEVICE_FINGERS_INFOS
,
paramList
);
JdbcExecuters
.
prepareBatchUpdate
(
Constant
.
SQL
.
DEVICE_FINGERS_INFOS
,
paramList
,
type
);
log
.
info
(
"充值记录清洗完成, uuid: {} , 入库大小: {} ,
ka: {} , 耗时: {} "
,
uuid
,
1
,
ka
,
stopwatch
.
stop
().
elapsed
(
TimeUnit
.
MILLISECONDS
));
log
.
info
(
"充值记录清洗完成, uuid: {} , 入库大小: {} ,
type: {} , ka: {} , 耗时: {} "
,
uuid
,
1
,
type
,
ka
,
stopwatch
.
stop
().
elapsed
(
TimeUnit
.
MILLISECONDS
));
}
...
...
qg-data-service/src/main/java/cn/quantgroup/qgdataservice/utils/JdbcExecuters.java
View file @
10ed45e2
...
...
@@ -73,7 +73,7 @@ public class JdbcExecuters {
}
}
public
static
void
prepareBatchUpdate
(
String
sql
,
List
<
List
<
Object
>>
paramList
)
{
public
static
void
prepareBatchUpdate
(
String
sql
,
List
<
List
<
Object
>>
paramList
,
String
type
)
{
Connection
conn
=
null
;
PreparedStatement
statement
=
null
;
try
{
...
...
@@ -96,15 +96,15 @@ public class JdbcExecuters {
}
statement
.
addBatch
();
}
catch
(
SQLException
e
)
{
log
.
error
(
"设备指纹清洗数据批量插入数据SQL异常 ,
param: {} "
,
JSON
.
toJSONString
(
param
),
e
);
log
.
error
(
"设备指纹清洗数据批量插入数据SQL异常 ,
type: {} , param: {} "
,
type
,
JSON
.
toJSONString
(
param
),
e
);
}
catch
(
Exception
e
)
{
log
.
error
(
"设备指纹清洗数据批量插入数据未知异常,
param: {} "
,
JSON
.
toJSONString
(
param
),
e
);
log
.
error
(
"设备指纹清洗数据批量插入数据未知异常,
type: {} , param: {} "
,
type
,
JSON
.
toJSONString
(
param
),
e
);
}
}
statement
.
executeBatch
();
conn
.
commit
();
}
catch
(
Exception
e
)
{
log
.
error
(
"设备指纹清洗数据批量插入数据未知异常,
sql: {} "
,
sql
,
e
);
log
.
error
(
"设备指纹清洗数据批量插入数据未知异常,
type: {} , sql: {} "
,
type
,
sql
,
e
);
}
finally
{
close
(
conn
,
statement
,
null
);
}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment