Join the social network of Tech Nerds, increase skill rank, get work, manage projects...
 
  • Copying file content to other file

    • 0
    • 0
    • 0
    • 0
    • 0
    • 0
    • 0
    • 98
    Answer it

    Hi,

    I have two files file1 an file2. I want to take each line from two files and compare with a keyword if the keyword matches in both the files. The complete line which matches from file1 should be written to file2 at the position below the matching line.

    File1 content:

    (Ingestion.py) test_Execution_Status_Update[Invalid_Hdfs_Log_Path_Test_Case]
    Moving log file from local temp '/workspace/ws_facilities_datalake/archibus/pytest_test/Ingestion/m_AB_TO_STG_bl_data_Ingestion_2024.log' of driver host 'ddlhdpdev08' to hdfs '/hivestore/dev/workspace/ws_facilities/archibus/bl1/stg_test1/'
    Log file move from local temp '/workspace/ws_facilities_datalake/archibus/pytest_test/Ingestion/m_AB_TO_STG_bl_data_Ingestion_2024.log' of driver host 'ddlhdpdev08' to hdfs '/hivestore/dev/workspace/ws_facilities/archibus/bl1/stg_test1/' failed and the error is Traceback (most recent call last):
     File "/workspace/ws_facilities_datalake/archibus/pytest_test/Ingestion/Ingestion.py", line 98, in Execution_Status_Update
      subprocess.check_output('hadoop fs -moveFromLocal -f {} {}'.format(Log_File,Hdfs_Log_Path), shell=True)
     File "/usr/lib64/python2.7/subprocess.py", line 575, in check_output
      raise CalledProcessError(retcode, cmd, output=output)
    CalledProcessError: Command 'hadoop fs -moveFromLocal -f /workspace/ws_facilities_datalake/archibus/pytest_test/Ingestion/m_AB_TO_STG_bl_data_Ingestion_2024.log /hivestore/dev/workspace/ws_facilities/archibus/bl1/stg_test1/' returned non-zero exit status 1
    
    
    (Ingestion.py) test_Execution_Status_Update[Valid_Hdfs_Log_Path_Test_Case0]
    Moving log file from local temp '/workspace/ws_facilities_datalake/archibus/pytest_test/Ingestion/m_AB_TO_STG_bl_data_Ingestion_2022.log' of driver host 'ddlhdpdev08' to hdfs '/hivestore/dev/workspace/ws_facilities/archibus/bl/stg_test'
    Log file move from local temp to hdfs is successful

     

    File2 Content:

    2021-01-06 02:59:46 (Ingestion.py:107) test_Execution_Status_Update[Invalid_Hdfs_Log_Path_Test_Case] [ERROR] Log file move from local temp '/workspace/ws_facilities_datalake/archibus/pytest/Ingestion/m_AB_TO_STG_bl_data_Ingestion_2024.log' of driver host 'ddlhdpdev08' to hdfs '/hivestore/dev/workspace/ws_facilities/archibus/bl1/stg_test1/' failed and the error is Traceback (most recent call last):
     File "/workspace/ws_facilities_datalake/archibus/pytest/Ingestion/Ingestion.py", line 98, in Execution_Status_Update
      subprocess.check_output('hadoop fs -moveFromLocal -f {} {}'.format(Log_File,Hdfs_Log_Path), shell=True)
     File "/usr/lib64/python2.7/subprocess.py", line 575, in check_output
      raise CalledProcessError(retcode, cmd, output=output)
    CalledProcessError: Command 'hadoop fs -moveFromLocal -f /workspace/ws_facilities_datalake/archibus/pytest/Ingestion/m_AB_TO_STG_bl_data_Ingestion_2024.log /hivestore/dev/workspace/ws_facilities/archibus/bl1/stg_test1/' returned non-zero exit status 1
    
    2021-01-06 02:59:50 (Ingestion.py:116) test_Source_Password_Decryption[Password_Decryption_Failure_Test_Case] [INFO] Reading password key file '/workspace/ws_facilities_datalake/archibus/pytest/Ingestion/archibus_user_dev.key'
    2021-01-06 02:59:50 (Ingestion.py:121) test_Source_Password_Decryption[Password_Decryption_Failure_Test_Case] [INFO] Password key file read is successful
    2021-01-06 02:59:50 (Ingestion.py:122) test_Source_Password_Decryption[Password_Decryption_Failure_Test_Case] [INFO]

     

    Here in my case the keyword that should match is:

    "test_Execution_Status_Update[Invalid_Hdfs_Log_Path_Test_Case] "

     

    Output Expected: File2

    2021-01-05 04:45:39 (Ingestion.py:107) test_Execution_Status_Update[Invalid_Hdfs_Log_Path_Test_Case] [ERROR] Log file move from local temp '/workspace/ws_facilities_datalake/archibus/pytest/Ingestion/m_AB_TO_STG_bl_data_Ingestion_2024.log' of driver host 'ddlhdpdev08' to hdfs '/hivestore/dev/workspace/ws_facilities/archibus/bl1/stg_test1/' failed and the error is Traceback (most recent call last):
     File "/workspace/ws_facilities_datalake/archibus/pytest/Ingestion/Ingestion.py", line 98, in Execution_Status_Update
      subprocess.check_output('hadoop fs -moveFromLocal -f {} {}'.format(Log_File,Hdfs_Log_Path), shell=True)
     File "/usr/lib64/python2.7/subprocess.py", line 575, in check_output
      raise CalledProcessError(retcode, cmd, output=output)
    CalledProcessError: Command 'hadoop fs -moveFromLocal -f /workspace/ws_facilities_datalake/archibus/pytest/Ingestion/m_AB_TO_STG_bl_data_Ingestion_2024.log /hivestore/dev/workspace/ws_facilities/archibus/bl1/stg_test1/' returned non-zero exit status 1
    (Ingestion.py) test_Execution_Status_Update[Invalid_Hdfs_Log_Path_Test_Case]
    Moving log file from local temp '/workspace/ws_facilities_datalake/archibus/pytest_test/Ingestion/m_AB_TO_STG_bl_data_Ingestion_2024.log' of driver host 'ddlhdpdev08' to hdfs '/hivestore/dev/workspace/ws_facilities/archibus/bl1/stg_test1/'
    Log file move from local temp '/workspace/ws_facilities_datalake/archibus/pytest_test/Ingestion/m_AB_TO_STG_bl_data_Ingestion_2024.log' of driver host 'ddlhdpdev08' to hdfs '/hivestore/dev/workspace/ws_facilities/archibus/bl1/stg_test1/' failed and the error is Traceback (most recent call last):
     File "/workspace/ws_facilities_datalake/archibus/pytest_test/Ingestion/Ingestion.py", line 98, in Execution_Status_Update
      subprocess.check_output('hadoop fs -moveFromLocal -f {} {}'.format(Log_File,Hdfs_Log_Path), shell=True)
     File "/usr/lib64/python2.7/subprocess.py", line 575, in check_output
      raise CalledProcessError(retcode, cmd, output=output)
    CalledProcessError: Command 'hadoop fs -moveFromLocal -f /workspace/ws_facilities_datalake/archibus/pytest_test/Ingestion/m_AB_TO_STG_bl_data_Ingestion_2024.log
    
    2021-01-05 04:45:42 (Ingestion.py:116) test_Source_Password_Decryption[Password_Decryption_Function_Failure_Test_Case] [INFO] Reading password key file '/workspace/ws_facilities_datalake/archibus/pytest/Ingestion/archibus_user_dev.key'
    2021-01-05 04:45:42 (Ingestion.py:121) test_Source_Password_Decryption[Password_Decryption_Function_Failure_Test_Case] [INFO] Password key file read is successful
    2021-01-05 04:45:42 (Ingestion.py:122) test_Source_Password_Decryption[Password_Decryption_Function_Failure_Test_Case] [INFO]
    2021-01-05 04:45:42 (Ingestion.py:129) test_Source_Password_Decryption[Password_Decryption_Function_Failure_Test_Case] [INFO] Reading encrypted password file '/workspace/ws_facilities_datalake/archibus/pytest/Ingestion/archibus_user_password_dev.encrypted'
    2021-01-05 04:45:42 (Ingestion.py:134) test_Source_Password_Decryption[Password_Decryption_Function_Failure_Test_Case] [INFO] Encrypted password file read is successful

     

 0 Answer(s)

Sign In
                           OR                           
                           OR                           
Register

Sign up using

                           OR                           
Forgot Password
Fill out the form below and instructions to reset your password will be emailed to you:
Reset Password
Fill out the form below and reset your password: