1 Introduction

This code shows the process of how we extract text-related features for sample paragraphs. Those features include text statistics, part-of-speech (POS) tags and syntactic structured features over parse trees (parse tree features). It is organised as follows. Section 2 loads libraries and sample paragraphs; section 3 is the process of extracting all textual features and POS tag features and Section 4 shows the process of how to extract parse tree features for each paragraph. Section 5 generates the final output table and the last session provides session information.

2 Preparation

2.1 Import libraries

Loading required packages. You may need to install some packages first if they are failed to be loaded.

load.lib<-c("xlsx", "dplyr","tidytext","ggplot2","ggthemes","wordcloud",
            "tm","stringr","e1071","ldatuning","pander","dplyr","pdftools",
            "stargazer","qdap","reshape","sqldf","stringi","stringr","reshape2",
            "readtext","zoo", "tibble", "lubridate", "data.table", "texreg",
            "MASS", "AER", "pscl","tidyr", "kableExtra", "textclean", "quanteda", 
            "tidytext", "tidyverse")

sapply(load.lib,require,character=TRUE)

mutate <- dplyr::mutate
select <- dplyr::select
count <- dplyr::count
summarise <- dplyr::summarise
rename <- plyr::rename

2.2 Import functions

Loading pre-defined functions that will be used later in this code.

source("./r_function/text_stats.R") 
source("./r_function/POS_tag_function.R")
source("./r_function/function_sentence_feature.R")

2.3 Data input

2.3.1 Import sample paragraphs

Input sample paragraphs.

#set up the folder to read sample paragraphs
text_file_path <- "../Survey Data/3_survey_group"
#read 5 surveys and combine them as one documents

file_list = list.files(text_file_path, pattern="*.csv")
file_MergedData <- do.call(rbind,lapply(paste(text_file_path, file_list, sep = "/"), read.csv))
#rename the column name and clean 
file_MergedData <- plyr::rename(file_MergedData, c("para" = "paragraph"))

2.3.2 Text clean

Clean text by replacing unrecognised characteristics such as “`”. This is an important step as those unrecognised characteristic will impact the accuracy of decomposing a paragraph into sentences. The distribution sample paragraphs by source is shown in the table below:

#replace non-English symbols in the text
file_MergedData$paragraph <- str_replace_all(file_MergedData$paragraph, "`","'")
file_MergedData$paragraph <- str_replace_all(file_MergedData$paragraph, "'","'")
file_MergedData$paragraph <- str_replace_all(file_MergedData$paragraph, "'","'")
# file_MergedData$paragraph <- str_replace_all(file_MergedData$paragraph, "??","'''")

#remove brackets and the contents between it
file_MergedData$paragraph <- bracketX(file_MergedData$paragraph)#remove the brackets and its contents in it

#remove extra white spaces
file_MergedData$paragraph <- rm_white_endmark(file_MergedData$paragraph) 
file_MergedData$paragraph <- rm_white_lead_trail(file_MergedData$paragraph)
file_MergedData$paragraph <- rm_white_multiple(file_MergedData$paragraph)
file_MergedData$paragraph <- rm_white_punctuation(file_MergedData$paragraph)
file_MergedData$paragraph <- rm_white_comma(file_MergedData$paragraph)

#create an index column for recording each row
file_MergedData$index <- 1:nrow(file_MergedData)

# file_MergedData %>% top_n(.,5) %>% kbl()

text_source_tb <- 
  file_MergedData %>% group_by(source) %>% summarise(n=n())%>% mutate(rel.freq = paste0(round(100 * n/sum(n), 0), "%"))

text_source_tb %>% 
  kbl(caption = "Composition of sample paragraph") %>%
  kable_classic(full_width = F, html_font = "Cambria")
Composition of sample paragraph
source n rel.freq
1_frs 50 5%
10_economist 200 20%
11_grattan 100 10%
2_bulletin 100 10%
3_rba_speeches 100 10%
4_smp_intro_2006_2019 100 10%
5_smp_main 50 5%
6_smp_boxes_06_19 100 10%
7_boe_main 50 5%
8_boe_ir_intro 50 5%
9_boe_speeches 100 10%

3 Extract textual feature

In this section, we generate the text related features all textual features, readability, argument features and syntactic features excluding structured synthetics feature over parse trees.

3.1 Text clean and preparation

Create a unique index number to record each paragraph. This variable will be used as a key to join other tables. The NLP package requires a large computing power, so running this code using the whole sample size may fail. Given the main purpose of this code is to show the process of generating text-related features, we limit the sample size to 10 paragraphs to ensure the code run smoothly.

## convert all factor variables to characters
file_MergedData %>% mutate_if(is.factor, as.character) -> file_MergedData
##create an unique index column
file_MergedData$question_index <- paste(file_MergedData$survey_group, file_MergedData$question_group, 
                                        file_MergedData$index, sep = "_")
#limited to 10 paragraphs to allow the following programs run smoothly
file_MergedData <- file_MergedData[1:10,]

3.2 Extract textual features

This code extracts textual features and readability features including:

  • Paragraph length
  • Sentence count
  • Number count
  • Comma count
  • Other punctuation count
  • First sentence with numbers
  • First sentence with “Table” or “Figure/Graph”
  • Syllables count
  • Average word length
  • Count of complicated words
  • FK grade level

A snapshot of the output is shown as below.

## create the feature list table
survey_feature1 <- para_stats_function(file_MergedData)
#A snapshot of the data is:
kbl(survey_feature1) %>%
  kable_paper() %>%
  scroll_box(width = "100%", height = "200px")
X year month issue paragraph source source_group survey_group question_group index question_index paragraph_clean word_count_stats sentence_count readability_stats.sylls readability_stats.polys fk_grade_level FRES_score comma_count punc_count digit_count
1 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. 1_frs G1 1 10 1 1_10_1 Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. 92 4 163 17 14.28652 33.60087 3 9 0
2 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. 1_frs G1 1 4 2 1_4_2 Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. 82 3 150 15 16.65537 24.33557 3 3 5
3 2017 October The Global Financial Environment | Financial Stability Review – October 2017 | RBA Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. 1_frs G1 1 9 3 1_9_3 Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. 25 1 54 8 19.64800 -1.27600 1 1 0
4 2016 October The Global Financial Environment | Financial Stability Review – October 2016 | RBA With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. 1_frs G1 1 3 4 1_3_4 With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. 73 3 143 20 17.01507 16.41338 5 4 0
5 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. 1_frs G1 1 3 5 1_3_5 If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. 89 3 163 21 17.59124 21.78176 3 6 0
6 2018 October The Global Financial Environment | Financial Stability Review – October 2018 | RBA Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. 1_frs G1 1 9 6 1_9_6 Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. 118 6 200 19 12.08000 43.48350 6 10 0
7 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. 1_frs G1 1 8 7 1_8_7 The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. 126 5 237 31 16.43324 22.12843 5 11 11
8 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. 1_frs G1 1 5 8 1_5_8 Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. 176 8 324 33 14.71273 28.76409 11 15 16
9 2019 October Household and Business Finances | Financial Stability Review – October 2019 | RBA Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. 1_frs G1 1 6 9 1_6_9 Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. 211 8 387 53 16.33890 24.89755 12 17 0
10 2016 April The Australian Financial System | Financial Stability Review – April 2016 | RBA Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. 1_frs G1 1 2 10 1_2_10 Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. 121 5 228 31 16.08271 22.86043 4 10 19

3.3 Extract POS features

This section extracts POS tag and argument features for a given pargaph and its sentences.

3.3.1 POS for a paragraph

Generate POS features for a given paragraph, including: * POS count * POS ratio A snapshot of the output is shown as below.

survey_feature2 <- word_pos_prop_function(survey_feature1) #calculate the POS taggers for each paragraph
# head(survey_feature2)

## calculate the proportions of each tag in the paragraph (pos tag count / total words)
para_pos_prop <-  round(survey_feature2[-1:-2]/rowSums(survey_feature2[-1:-2]),4)*100

colnames(para_pos_prop) <- paste("pos_prop",colnames(para_pos_prop), sep = "_")

survey_feature2 <- cbind(survey_feature2,para_pos_prop)

survey_feature_part2 <- left_join(survey_feature1,survey_feature2, by="question_index")

#A snapshot of the output from this step is:
kbl(survey_feature_part2) %>%
  kable_paper() %>%
  scroll_box(width = "100%", height = "200px")
X year month issue paragraph source source_group survey_group question_group index.x question_index paragraph_clean word_count_stats sentence_count readability_stats.sylls readability_stats.polys fk_grade_level FRES_score comma_count punc_count digit_count index.y CC DT IN JJ JJR MD NN NNS PRP PRP$ RB RBR TO VB VBD VBG VBN VBP VBZ WDT CD RBS POS RP JJS NNP EX pos_prop_CC pos_prop_DT pos_prop_IN pos_prop_JJ pos_prop_JJR pos_prop_MD pos_prop_NN pos_prop_NNS pos_prop_PRP pos_prop_PRP$ pos_prop_RB pos_prop_RBR pos_prop_TO pos_prop_VB pos_prop_VBD pos_prop_VBG pos_prop_VBN pos_prop_VBP pos_prop_VBZ pos_prop_WDT pos_prop_CD pos_prop_RBS pos_prop_POS pos_prop_RP pos_prop_JJS pos_prop_NNP pos_prop_EX
1 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. 1_frs G1 1 10 1 1_10_1 Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. 92 4 163 17 14.28652 33.60087 3 9 0 1 5 6 8 13 1 3 14 12 2 1 5 1 4 6 2 1 2 3 1 1 0 0 0 0 0 0 0 5.49 6.59 8.79 14.29 1.10 3.30 15.38 13.19 2.20 1.10 5.49 1.10 4.40 6.59 2.20 1.10 2.20 3.30 1.10 1.10 0.00 0.00 0.00 0.00 0.00 0.00 0.00
2 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. 1_frs G1 1 4 2 1_4_2 Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. 82 3 150 15 16.65537 24.33557 3 3 5 2 2 6 13 11 1 2 16 12 1 0 5 1 2 3 2 0 2 2 1 0 2 0 0 0 0 0 0 2.38 7.14 15.48 13.10 1.19 2.38 19.05 14.29 1.19 0.00 5.95 1.19 2.38 3.57 2.38 0.00 2.38 2.38 1.19 0.00 2.38 0.00 0.00 0.00 0.00 0.00 0.00
3 2017 October The Global Financial Environment | Financial Stability Review – October 2017 | RBA Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. 1_frs G1 1 9 3 1_9_3 Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. 25 1 54 8 19.64800 -1.27600 1 1 0 3 1 1 3 4 1 0 1 4 0 0 2 0 1 2 0 2 1 1 0 0 0 1 0 0 0 0 0 4.00 4.00 12.00 16.00 4.00 0.00 4.00 16.00 0.00 0.00 8.00 0.00 4.00 8.00 0.00 8.00 4.00 4.00 0.00 0.00 0.00 4.00 0.00 0.00 0.00 0.00 0.00
4 2016 October The Global Financial Environment | Financial Stability Review – October 2016 | RBA With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. 1_frs G1 1 3 4 1_3_4 With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. 73 3 143 20 17.01507 16.41338 5 4 0 4 4 5 10 12 0 0 14 11 0 0 2 0 3 1 0 3 2 3 2 0 0 0 1 1 0 0 0 5.41 6.76 13.51 16.22 0.00 0.00 18.92 14.86 0.00 0.00 2.70 0.00 4.05 1.35 0.00 4.05 2.70 4.05 2.70 0.00 0.00 0.00 1.35 1.35 0.00 0.00 0.00
5 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. 1_frs G1 1 3 5 1_3_5 If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. 89 3 163 21 17.59124 21.78176 3 6 0 5 5 3 15 16 0 2 15 11 2 0 5 0 2 2 0 2 1 5 1 0 0 0 1 1 0 0 0 5.62 3.37 16.85 17.98 0.00 2.25 16.85 12.36 2.25 0.00 5.62 0.00 2.25 2.25 0.00 2.25 1.12 5.62 1.12 0.00 0.00 0.00 1.12 1.12 0.00 0.00 0.00
6 2018 October The Global Financial Environment | Financial Stability Review – October 2018 | RBA Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. 1_frs G1 1 9 6 1_9_6 Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. 118 6 200 19 12.08000 43.48350 6 10 0 6 8 14 14 9 1 2 28 10 3 0 2 1 6 4 1 1 6 2 4 1 0 1 0 0 0 0 0 6.78 11.86 11.86 7.63 0.85 1.69 23.73 8.47 2.54 0.00 1.69 0.85 5.08 3.39 0.85 0.85 5.08 1.69 3.39 0.85 0.00 0.85 0.00 0.00 0.00 0.00 0.00
7 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. 1_frs G1 1 8 7 1_8_7 The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. 126 5 237 31 16.43324 22.12843 5 11 11 7 2 18 18 13 1 2 33 9 0 2 5 0 4 4 3 1 6 0 5 0 5 0 4 0 0 0 0 1.48 13.33 13.33 9.63 0.74 1.48 24.44 6.67 0.00 1.48 3.70 0.00 2.96 2.96 2.22 0.74 4.44 0.00 3.70 0.00 3.70 0.00 2.96 0.00 0.00 0.00 0.00
8 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. 1_frs G1 1 5 8 1_5_8 Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. 176 8 324 33 14.71273 28.76409 11 15 16 8 8 11 27 24 2 1 36 21 0 4 9 1 4 2 1 4 9 4 2 1 4 0 0 0 2 1 0 4.49 6.18 15.17 13.48 1.12 0.56 20.22 11.80 0.00 2.25 5.06 0.56 2.25 1.12 0.56 2.25 5.06 2.25 1.12 0.56 2.25 0.00 0.00 0.00 1.12 0.56 0.00
9 2019 October Household and Business Finances | Financial Stability Review – October 2019 | RBA Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. 1_frs G1 1 6 9 1_6_9 Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. 211 8 387 53 16.33890 24.89755 12 17 0 9 9 15 28 28 2 3 39 32 1 0 7 2 7 7 0 10 5 11 2 2 0 0 0 0 0 0 1 4.27 7.11 13.27 13.27 0.95 1.42 18.48 15.17 0.47 0.00 3.32 0.95 3.32 3.32 0.00 4.74 2.37 5.21 0.95 0.95 0.00 0.00 0.00 0.00 0.00 0.00 0.47
10 2016 April The Australian Financial System | Financial Stability Review – April 2016 | RBA Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. 1_frs G1 1 2 10 1_2_10 Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. 121 5 228 31 16.08271 22.86043 4 10 19 10 1 19 12 9 0 1 35 5 0 2 4 0 11 6 3 3 5 1 4 0 7 0 2 0 0 0 0 0.77 14.62 9.23 6.92 0.00 0.77 26.92 3.85 0.00 1.54 3.08 0.00 8.46 4.62 2.31 2.31 3.85 0.77 3.08 0.00 5.38 0.00 1.54 0.00 0.00 0.00 0.00

3.3.2 POS Tagger features for sentences

Decompose each paragraph into sentences to generate sentence-related POS features, including:

  • POS count in the first sentence
  • POS ratio in the first sentence
  • POS ratio in the last sentence
  • POS count in the last sentence

(1) Decompose paragraph into sentences

Extract sentences for a given paragraph using unnest_tokens() function, and generaing three output table:

  • text_sentence: record sentences for each paragraph
  • first_sentence: record the first sentence of each paragrpah
  • last_sentence: record the last sentence of each paragraph
## untoken paragraphs to sentences and rank each sentence 
text_sentence <- file_MergedData %>% 
  unnest_tokens(sentence, paragraph, token = "sentences") %>%
  group_by(question_index) %>% 
  mutate(rank = c(1:n())) #tokenize to sentence and add a column called 'rank' to extract the first sentence of each group.

## sentence feature 1: check if the first sentence include words "table", "figure", "graph" or include numbers
  
first_sentence <- text_sentence %>% 
  select(question_index, sentence, rank) %>% 
  filter(rank == 1) %>% 
  mutate(tf_index = ifelse(str_detect(sentence, "\\table\\b|graph|chart|figure"),1,0),
           number_index = ifelse(str_detect(sentence, "[0123456789]"),1,0)) #check if the first sentence include table, graph, chart or figure - those would indicate that the dicussion of the whole sentence will be based on facts;

first_sentence$sentence <- gsub("&", "and", first_sentence$sentence) #replace & with "and"

### extract the last sentence
last_sentence <- text_sentence %>%
  group_by(question_index) %>%
  filter(rank == max(rank)) %>%
  select(question_index, sentence, rank)

last_sentence$sentence <- gsub("&", "and", last_sentence$sentence) #replace & with "and"

#A snapshot of the output table of the text_setence table
kbl(text_sentence) %>%
  kable_paper() %>%
  scroll_box(width = "100%", height = "200px")
X year month issue source source_group survey_group question_group index question_index sentence rank
1 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 10 1 1_10_1 along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. 1
1 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 10 1 1_10_1 this has increased their liquidity risks and made them even more interconnected and systemic. 2
1 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 10 1 1_10_1 if corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. 3
1 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 10 1 1_10_1 it could also transmit distress to other institutions that investors consider to have a similar vulnerability. 4
2 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA 1_frs G1 1 4 2 1_4_2 total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. 1
2 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA 1_frs G1 1 4 2 1_4_2 while net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. 2
2 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA 1_frs G1 1 4 2 1_4_2 superannuation funds will therefore need to consider the associated liquidity implications. 3
3 2017 October The Global Financial Environment | Financial Stability Review – October 2017 | RBA 1_frs G1 1 9 3 1_9_3 despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. 1
4 2016 October The Global Financial Environment | Financial Stability Review – October 2016 | RBA 1_frs G1 1 3 4 1_3_4 with the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. 1
4 2016 October The Global Financial Environment | Financial Stability Review – October 2016 | RBA 1_frs G1 1 3 4 1_3_4 as for china, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. 2
4 2016 October The Global Financial Environment | Financial Stability Review – October 2016 | RBA 1_frs G1 1 3 4 1_3_4 lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. 3
5 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 3 5 1_3_5 if financial strains that threaten growth in china emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. 1
5 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 3 5 1_3_5 direct financial linkages between china and other economies are small in aggregate because china’s capital account is still relatively closed. 2
5 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 3 5 1_3_5 but these linkages have grown – both in terms of foreign bank lending to china and chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. 3
6 2018 October The Global Financial Environment | Financial Stability Review – October 2018 | RBA 1_frs G1 1 9 6 1_9_6 much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent nbfis. 1
6 2018 October The Global Financial Environment | Financial Stability Review – October 2018 | RBA 1_frs G1 1 9 6 1_9_6 most of this lending is ultimately funded by the banking sector. 2
6 2018 October The Global Financial Environment | Financial Stability Review – October 2018 | RBA 1_frs G1 1 9 6 1_9_6 while this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. 3
6 2018 October The Global Financial Environment | Financial Stability Review – October 2018 | RBA 1_frs G1 1 9 6 1_9_6 the riskier nature of the lending, and the obscure and complex interconnections between nbfis and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. 4
6 2018 October The Global Financial Environment | Financial Stability Review – October 2018 | RBA 1_frs G1 1 9 6 1_9_6 loan losses and defaults have been modest to date. 5
6 2018 October The Global Financial Environment | Financial Stability Review – October 2018 | RBA 1_frs G1 1 9 6 1_9_6 but if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. 6
7 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA 1_frs G1 1 8 7 1_8_7 the increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. 1
7 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA 1_frs G1 1 8 7 1_8_7 the leverage ratio is a non-risk based measure of a bank’s tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. 2
7 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA 1_frs G1 1 8 7 1_8_7 the leverage ratio framework is yet to be finalised internationally, although the basel committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from january 2018. 3
7 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA 1_frs G1 1 8 7 1_8_7 each of the major banks’ leverage ratios was around 5 per cent at june 2016, well above that minimum. 4
7 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA 1_frs G1 1 8 7 1_8_7 at this level, the major australian banks’ leverage ratio sits around the median of international banks. 5
8 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 5 8 1_5_8 housing market risks are also present in some emerging market and asian economies. 1
8 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 5 8 1_5_8 this reflects large increases in residential property prices over recent years – including in hong kong, brazil, malaysia, taiwan and turkey – alongside increased household indebtedness. 2
8 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 5 8 1_5_8 price growth has moderated more recently and prices have fallen in some economies, including brazil, russia and taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. 3
8 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 5 8 1_5_8 housing prices in hong kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. 4
8 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 5 8 1_5_8 but prices have fallen recently amid concerns about economic conditions in china and slower credit growth. 5
8 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 5 8 1_5_8 housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. 6
8 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA 1_frs G1 1 5 8 1_5_8 despite the slowdown in the housing market, the hong kong monetary authority imposed a countercyclical capital buffer of 0.625 per cent in january 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-gdp and housing prices-to-rents relative to their long-run trends. 7
9 2019 October Household and Business Finances | Financial Stability Review – October 2019 | RBA 1_frs G1 1 6 9 1_6_9 personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. 1
9 2019 October Household and Business Finances | Financial Stability Review – October 2019 | RBA 1_frs G1 1 6 9 1_6_9 in recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. 2
9 2019 October Household and Business Finances | Financial Stability Review – October 2019 | RBA 1_frs G1 1 6 9 1_6_9 in part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. 3
9 2019 October Household and Business Finances | Financial Stability Review – October 2019 | RBA 1_frs G1 1 6 9 1_6_9 more recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. 4
9 2019 October Household and Business Finances | Financial Stability Review – October 2019 | RBA 1_frs G1 1 6 9 1_6_9 buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. 5
9 2019 October Household and Business Finances | Financial Stability Review – October 2019 | RBA 1_frs G1 1 6 9 1_6_9 while these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. 6
9 2019 October Household and Business Finances | Financial Stability Review – October 2019 | RBA 1_frs G1 1 6 9 1_6_9 however, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. 7
9 2019 October Household and Business Finances | Financial Stability Review – October 2019 | RBA 1_frs G1 1 6 9 1_6_9 this could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. 8
10 2016 April The Australian Financial System | Financial Stability Review – April 2016 | RBA 1_frs G1 1 2 10 1_2_10 australian banks using the irb approach to credit risk have been required to disclose their leverage ratio from mid 2015. 1
10 2016 April The Australian Financial System | Financial Stability Review – April 2016 | RBA 1_frs G1 1 2 10 1_2_10 the leverage ratio is a non-risk-based measure of a bank’s tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. 2
10 2016 April The Australian Financial System | Financial Stability Review – April 2016 | RBA 1_frs G1 1 2 10 1_2_10 the leverage ratio framework is yet to be finalised internationally, although the basel committee’s governing body agreed the minimum requirement should be 3 per cent. 3
10 2016 April The Australian Financial System | Financial Stability Review – April 2016 | RBA 1_frs G1 1 2 10 1_2_10 the basel committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from january 2018. 4
10 2016 April The Australian Financial System | Financial Stability Review – April 2016 | RBA 1_frs G1 1 2 10 1_2_10 each of the australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at december 2015, well above the minimum. 5

(2) Extract POS features for the frist and last sentence of each paragraphs

## the first sentence POS tag features
first_sentence_pos <- sentence_pos_function(first_sentence) %>% as.data.frame()

colnames(first_sentence_pos)[3:ncol(first_sentence_pos)] <- paste("sent_1st",colnames(first_sentence_pos)[3:ncol(first_sentence_pos)], sep = "_")


## the last sentence POS tag features
last_sentence_pos <- sentence_pos_function(last_sentence) %>% as.data.frame()

colnames(last_sentence_pos)[3:ncol(last_sentence_pos)] <- paste("sent_last",colnames(last_sentence_pos)[3:ncol(last_sentence_pos)], sep = "_")

sentence_pos <- left_join(first_sentence_pos, last_sentence_pos, by = "question_index")

sentence_pos %>% head() %>% kbl() %>%
  kable_paper() %>%
  scroll_box(width = "100%", height = "200px")
index.x question_index sent_1st_CC sent_1st_DT sent_1st_IN sent_1st_JJ sent_1st_JJR sent_1st_NN sent_1st_NNS sent_1st_RB sent_1st_VBN sent_1st_VBP sent_1st_CD sent_1st_VBD sent_1st_RBS sent_1st_TO sent_1st_VB sent_1st_VBG sent_1st_RP sent_1st_VBZ sent_1st_MD sent_1st_PRP sent_1st_WDT sent_1st_RBR sent_1st_PRP$ sent_1st_prop_CC sent_1st_prop_DT sent_1st_prop_IN sent_1st_prop_JJ sent_1st_prop_JJR sent_1st_prop_NN sent_1st_prop_NNS sent_1st_prop_RB sent_1st_prop_VBN sent_1st_prop_VBP sent_1st_prop_CD sent_1st_prop_VBD sent_1st_prop_RBS sent_1st_prop_TO sent_1st_prop_VB sent_1st_prop_VBG sent_1st_prop_RP sent_1st_prop_VBZ sent_1st_prop_MD sent_1st_prop_PRP sent_1st_prop_WDT sent_1st_prop_RBR sent_1st_prop_PRP$ index.y sent_last_DT sent_last_JJ sent_last_MD sent_last_NN sent_last_NNS sent_last_PRP sent_last_RB sent_last_TO sent_last_VB sent_last_VBP sent_last_WDT sent_last_VBN sent_last_CC sent_last_IN sent_last_JJR sent_last_RBS sent_last_VBG sent_last_POS sent_last_VBZ sent_last_VBD sent_last_CD sent_last_PRP$ sent_last_RBR sent_last_prop_DT sent_last_prop_JJ sent_last_prop_MD sent_last_prop_NN sent_last_prop_NNS sent_last_prop_PRP sent_last_prop_RB sent_last_prop_TO sent_last_prop_VB sent_last_prop_VBP sent_last_prop_WDT sent_last_prop_VBN sent_last_prop_CC sent_last_prop_IN sent_last_prop_JJR sent_last_prop_RBS sent_last_prop_VBG sent_last_prop_POS sent_last_prop_VBZ sent_last_prop_VBD sent_last_prop_CD sent_last_prop_PRP$ sent_last_prop_RBR
1 1_10_1 1 2 5 4 1 6 3 2 1 2 0 0 0 0 0 0 0 0 0 0 0 0 0 3.70 7.41 18.52 14.81 3.70 22.22 11.11 7.41 3.70 7.41 0.00 0.00 0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0 1 1 2 1 2 2 1 1 2 2 1 1 0 0 0 0 0 0 0 0 0 0 0 0 6.25 12.50 6.25 12.50 12.50 6.25 6.25 12.50 12.50 6.25 6.25 0.00 0.00 0.00 0 0 0.00 0.00 0.00 0.00 0 0 0
1 1_4_2 1 3 9 7 0 10 4 3 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 2.44 7.32 21.95 17.07 0.00 24.39 9.76 7.32 0.00 0.00 4.88 4.88 0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0 1 1 0 1 2 2 0 1 1 2 0 0 1 0 0 0 0 0 0 0 0 0 0 0 9.09 0.00 9.09 18.18 18.18 0.00 9.09 9.09 18.18 0.00 0.00 9.09 0.00 0.00 0 0 0.00 0.00 0.00 0.00 0 0 0
1 1_9_3 1 1 3 4 1 1 4 2 1 1 0 0 1 1 2 2 0 0 0 0 0 0 0 4.00 4.00 12.00 16.00 4.00 4.00 16.00 8.00 4.00 4.00 0.00 0.00 4 4.00 8.00 8.00 0.00 0.00 0.00 0.00 0.00 0.00 0 1 1 4 0 1 4 0 2 1 2 1 0 1 1 3 1 1 2 0 0 0 0 0 0 4.00 16.00 0.00 4.00 16.00 0.00 8.00 4.00 8.00 4.00 0.00 4.00 4.00 12.00 4 4 8.00 0.00 0.00 0.00 0 0 0
1 1_3_4 2 3 4 3 0 6 2 0 1 0 0 0 0 2 1 2 1 1 0 0 0 0 0 7.14 10.71 14.29 10.71 0.00 21.43 7.14 0.00 3.57 0.00 0.00 0.00 0 7.14 3.57 7.14 3.57 3.57 0.00 0.00 0.00 0.00 0 1 2 5 0 2 5 0 2 1 0 2 0 1 1 3 0 0 2 1 1 0 0 0 0 7.14 17.86 0.00 7.14 17.86 0.00 7.14 3.57 0.00 7.14 0.00 3.57 3.57 10.71 0 0 7.14 3.57 3.57 0.00 0 0 0
1 1_3_5 1 0 5 4 0 5 5 2 0 2 0 0 0 1 1 1 1 0 1 1 1 0 0 3.23 0.00 16.13 12.90 0.00 16.13 16.13 6.45 0.00 6.45 0.00 0.00 0 3.23 3.23 3.23 3.23 0.00 3.23 3.23 3.23 0.00 0 1 3 6 1 6 4 1 1 1 1 2 0 1 3 6 0 0 1 0 0 0 0 0 0 8.11 16.22 2.70 16.22 10.81 2.70 2.70 2.70 2.70 5.41 0.00 2.70 8.11 16.22 0 0 2.70 0.00 0.00 0.00 0 0 0
1 1_9_6 1 3 4 2 1 5 0 1 2 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4.76 14.29 19.05 9.52 4.76 23.81 0.00 4.76 9.52 0.00 0.00 0.00 0 0.00 0.00 0.00 0.00 4.76 0.00 0.00 0.00 4.76 0 1 2 2 2 3 1 2 0 1 3 0 1 0 1 4 0 0 0 0 0 1 0 0 0 8.70 8.70 8.70 13.04 4.35 8.70 0.00 4.35 13.04 0.00 4.35 0.00 4.35 17.39 0 0 0.00 0.00 0.00 4.35 0 0 0

(3) POS feature for the first 3 words in the 1st sentence

Extract POS features for the first three words in the openning sentence of a given paragraph.

#this is the function to remove the apostrophes that are used to quote a sing word, for example 'peaking', but will keep the apostrophe that is used in the middle of a work, for example ABS's forecast, etc.
fun1 <- function(x) substr(x, 1 + (1 * as.numeric(substr(x,1,1)=="'")), nchar(x) - (1 * as.numeric(substr(x, nchar(x), nchar(x)) == "'")))
word_pos_function <- function(text_input_df, variable_choose){
  
   text_data <- text_input_df %>% select(question_index, variable_choose)
   
   #rename the selected word to word
   colnames(text_data)[2] <- "word_choose"
   
   text_p1 <- text_data[str_detect(text_data$word_choose, "'"),]
   
   if(nrow(text_p1)==0){
    text_p1 <- NULL
    } else {
     for (i in (1:nrow(text_p1))){
      text_p1$word_pos[i] <- paste(
        tagPOS(text_p1$word_choose[i])$POStags[1],
        tagPOS(text_p1$word_choose[i])$POStags[2], sep = "_")
     }
      }
  
  text_p2 <- text_data[!str_detect(text_data$word_choose, "'"),]
  
  text_p2$word_pos <- tagPOS(text_p2$word_choose)$POStags

  word_pos_result <- rbind(text_p1,text_p2)
  word_pos_result <- word_pos_result %>% arrange(as.numeric(row.names(word_pos_result)))
  
  word_pos_result

 }


sentence_text <- first_sentence

  
  first3words <- cbind(question_index = sentence_text$question_index,
                       start_word = word(sentence_text$sentence,1), 
                       second_word =word(sentence_text$sentence, 2), 
                       third_word = word(sentence_text$sentence, 3)) %>% as.data.frame()

  first3words$start_word <- fun1(as.character(first3words$start_word))
  first3words$second_word <- fun1(as.character(first3words$second_word))
  first3words$third_word <- fun1(as.character(first3words$third_word))
  
  # first3words$start_word <- removePunctuation(as.character(first3words$start_word))
  # first3words$second_word <- removePunctuation(as.character(first3words$second_word))
  # first3words$third_word <- removePunctuation(as.character(first3words$third_word))
  
  
  #remove all punctuations except for the apostrophe (')
  first3words$start_word <- gsub("[^[:alnum:][:space:]']", "", first3words$start_word)
  first3words$second_word <- gsub("[^[:alnum:][:space:]']", "", first3words$second_word)
  first3words$third_word <- gsub("[^[:alnum:][:space:]']", "", first3words$third_word)
 
  
 
library(NLP)
library(openNLP)
 

first_word_pos_df <- word_pos_function(first3words,  "start_word")

second_word_pos_df <- word_pos_function(first3words,  "second_word")


third_word_pos_df <- word_pos_function(first3words,  "third_word")



first3words_pos_df <- 
  left_join (first3words, first_word_pos_df, by = "question_index") %>% 
  left_join(.,second_word_pos_df, by = "question_index") %>% 
  left_join(.,third_word_pos_df, by = "question_index")


first3words_pos_df <- rename(first3words_pos_df, 
                             c("word_pos.x" = "word_pos.word1", "word_pos.y" = "word_pos.word2", "word_pos" = "word_pos.word3"))

#A snapshot of the output table is:
kbl(first3words_pos_df) %>%
  kable_paper() %>%
  scroll_box(width = "100%", height = "200px")
question_index start_word second_word third_word word_choose.x word_pos.word1 word_choose.y word_pos.word2 word_choose word_pos.word3
1_10_1 along with the along IN with IN the DT
1_4_2 total superannuation assets total JJ superannuation NN assets NNS
1_9_3 despite challenging economic despite IN challenging VBG economic JJ
1_3_4 with the increasing with IN the DT increasing VBG
1_3_5 if financial strains if IN financial JJ strains NNS
1_9_6 much of the much JJ of IN the DT
1_8_7 the increase in the DT increase NN in IN
1_5_8 housing market risks housing NN market NN risks NNS
1_6_9 personal debt which personal JJ debt NN which WDT
1_2_10 australian banks using australian NN banks NNS using VBG

(4) POS feature for the fist words of each sentence

Extract the POS features for the first word of each setence in a given paragraph.

## extract the first word of first sentence
firstwords <- cbind(question_index = text_sentence$question_index, rank = text_sentence$rank,
                       start_word = word(text_sentence$sentence,1)) %>% as.data.frame() #4196 rows

#remove double apostrophe (such as 'yes')
firstwords$start_word <- fun1(as.character(firstwords$start_word))
  
#remove all punctuations except for the apostrophe (')
firstwords$start_word <- gsub("[^[:alnum:][:space:]']", "", firstwords$start_word)


text_input_df <- firstwords
variable_choose <- "start_word"
  
text_data <- text_input_df %>% select(question_index, rank, variable_choose) %>% filter(nchar(start_word) > 0)
  
#rename the selected word to word_choose
colnames(text_data)[3] <- "word_choose"

text_p1 <- text_data[str_detect(text_data$word_choose, "'"),] #find the sentences beginning with "'"
   
if(nrow(text_p1)==0){
    text_p1 <- NULL
    } else {
     for (i in (1:nrow(text_p1))){
      text_p1$word_pos[i] <- paste(
        tagPOS(text_p1$word_choose[i])$POStags[1],
        tagPOS(text_p1$word_choose[i])$POStags[2], sep = "_")
     }
      }
  
text_p2 <- text_data[!str_detect(text_data$word_choose, "'"),] 
  
text_p2$word_pos <- tagPOS(text_p2$word_choose)$POStags
  
word_pos_result <- rbind(text_p1,text_p2)
word_pos_result <- word_pos_result %>% arrange(as.numeric(row.names(word_pos_result)))
  
firstwordsall <- word_pos_result
 
## count the PoS tag for the first word of each sentence
  
allsent_firstwords <- firstwordsall %>% select(-word_choose, -rank) %>% group_by(question_index) %>% count(word_pos) %>% spread(word_pos, n)

colnames(allsent_firstwords)[2:ncol(allsent_firstwords)] <- paste("sent_1st_word",colnames(allsent_firstwords)[2:ncol(allsent_firstwords)], sep = "_")

#replacing NAs with 0
allsent_firstwords[is.na(allsent_firstwords)] <- 0

#A snapshot of the output table is:
kbl(allsent_firstwords) %>%
  kable_paper() %>%
  scroll_box(width = "100%", height = "200px")
question_index sent_1st_word_CC sent_1st_word_DT sent_1st_word_IN sent_1st_word_JJ sent_1st_word_JJR sent_1st_word_JJS sent_1st_word_NN sent_1st_word_PRP sent_1st_word_RB
1_10_1 0 1 2 0 0 0 0 1 0
1_2_10 0 4 0 1 0 0 0 0 0
1_3_4 0 0 2 0 0 0 1 0 0
1_3_5 1 0 1 1 0 0 0 0 0
1_4_2 0 0 1 1 0 0 1 0 0
1_5_8 1 1 1 0 0 0 4 0 0
1_6_9 0 1 3 1 1 0 1 0 1
1_8_7 0 4 1 0 0 0 0 0 0
1_9_3 0 0 1 0 0 0 0 0 0
1_9_6 1 1 1 1 0 1 1 0 0

3.4 Extracting argument features

Using a list of clue words listed in Robin Cohen (1984) to generate argumentative features including:

  • Count of each type of clue words
  • Count of clue words in the first sentence
  • Count of clue words in the last sentence of a paragraph

A snapshot of the output table is shown as below.

clue_words <- read.csv("./data_input/clue_words.csv")

sentence_clue_feature <- clue_words_feature_function(text_sentence) %>% as.data.frame()


text_feature_pos_clue <- 
  left_join(survey_feature_part2, first_word_pos_df, by = "question_index") %>% 
  left_join(., first3words_pos_df, by = "question_index") %>%
  left_join(.,sentence_clue_feature, by = "question_index") %>% 
  left_join(.,allsent_firstwords, by = "question_index") %>%
  select(-start_word,-second_word, -third_word) #CC.X is for the whole paragraph; CC.y is for the sentence 1 features of POS (count).

#A snapshot of the output table is:
kbl(text_feature_pos_clue) %>%
  kable_paper() %>%
  scroll_box(width = "100%", height = "200px")
X year month issue paragraph source source_group survey_group question_group index.x question_index paragraph_clean word_count_stats sentence_count readability_stats.sylls readability_stats.polys fk_grade_level FRES_score comma_count punc_count digit_count index.y CC DT IN JJ JJR MD NN NNS PRP PRP$ RB RBR TO VB VBD VBG VBN VBP VBZ WDT CD RBS POS RP JJS NNP EX pos_prop_CC pos_prop_DT pos_prop_IN pos_prop_JJ pos_prop_JJR pos_prop_MD pos_prop_NN pos_prop_NNS pos_prop_PRP pos_prop_PRP$ pos_prop_RB pos_prop_RBR pos_prop_TO pos_prop_VB pos_prop_VBD pos_prop_VBG pos_prop_VBN pos_prop_VBP pos_prop_VBZ pos_prop_WDT pos_prop_CD pos_prop_RBS pos_prop_POS pos_prop_RP pos_prop_JJS pos_prop_NNP pos_prop_EX word_choose.x.x word_pos word_choose.x word_pos.word1 word_choose.y word_pos.word2 word_choose.y.y word_pos.word3 sent1st_clue_Attitudinal sent1st_clue_connective sentlast_clue_Attitudinal sentlast_clue_connective sentlast_clue_Contrast sentlast_clue_summary sentmiddle_clue_Attitudinal sentmiddle_clue_connective sentmiddle_clue_detail sentmiddle_clue_inference sent_1st_word_CC sent_1st_word_DT sent_1st_word_IN sent_1st_word_JJ sent_1st_word_JJR sent_1st_word_JJS sent_1st_word_NN sent_1st_word_PRP sent_1st_word_RB
1 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. 1_frs G1 1 10 1 1_10_1 Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. 92 4 163 17 14.28652 33.60087 3 9 0 1 5 6 8 13 1 3 14 12 2 1 5 1 4 6 2 1 2 3 1 1 0 0 0 0 0 0 0 5.49 6.59 8.79 14.29 1.10 3.30 15.38 13.19 2.20 1.10 5.49 1.10 4.40 6.59 2.20 1.10 2.20 3.30 1.10 1.10 0.00 0.00 0.00 0.00 0.00 0.00 0.00 along IN along IN with IN the DT 1 1 0 1 0 0 0 0 0 0 0 1 2 0 0 0 0 1 0
2 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. 1_frs G1 1 4 2 1_4_2 Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. 82 3 150 15 16.65537 24.33557 3 3 5 2 2 6 13 11 1 2 16 12 1 0 5 1 2 3 2 0 2 2 1 0 2 0 0 0 0 0 0 2.38 7.14 15.48 13.10 1.19 2.38 19.05 14.29 1.19 0.00 5.95 1.19 2.38 3.57 2.38 0.00 2.38 2.38 1.19 0.00 2.38 0.00 0.00 0.00 0.00 0.00 0.00 total JJ total JJ superannuation NN assets NNS 2 0 0 0 0 1 0 0 0 0 0 0 1 1 0 0 1 0 0
3 2017 October The Global Financial Environment | Financial Stability Review – October 2017 | RBA Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. 1_frs G1 1 9 3 1_9_3 Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. 25 1 54 8 19.64800 -1.27600 1 1 0 3 1 1 3 4 1 0 1 4 0 0 2 0 1 2 0 2 1 1 0 0 0 1 0 0 0 0 0 4.00 4.00 12.00 16.00 4.00 0.00 4.00 16.00 0.00 0.00 8.00 0.00 4.00 8.00 0.00 8.00 4.00 4.00 0.00 0.00 0.00 4.00 0.00 0.00 0.00 0.00 0.00 despite IN despite IN challenging VBG economic JJ NA NA NA NA NA NA NA NA NA NA 0 0 1 0 0 0 0 0 0
4 2016 October The Global Financial Environment | Financial Stability Review – October 2016 | RBA With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. 1_frs G1 1 3 4 1_3_4 With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. 73 3 143 20 17.01507 16.41338 5 4 0 4 4 5 10 12 0 0 14 11 0 0 2 0 3 1 0 3 2 3 2 0 0 0 1 1 0 0 0 5.41 6.76 13.51 16.22 0.00 0.00 18.92 14.86 0.00 0.00 2.70 0.00 4.05 1.35 0.00 4.05 2.70 4.05 2.70 0.00 0.00 0.00 1.35 1.35 0.00 0.00 0.00 with IN with IN the DT increasing VBG NA NA NA NA NA NA NA NA NA NA 0 0 2 0 0 0 1 0 0
5 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. 1_frs G1 1 3 5 1_3_5 If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. 89 3 163 21 17.59124 21.78176 3 6 0 5 5 3 15 16 0 2 15 11 2 0 5 0 2 2 0 2 1 5 1 0 0 0 1 1 0 0 0 5.62 3.37 16.85 17.98 0.00 2.25 16.85 12.36 2.25 0.00 5.62 0.00 2.25 2.25 0.00 2.25 1.12 5.62 1.12 0.00 0.00 0.00 1.12 1.12 0.00 0.00 0.00 if IN if IN financial JJ strains NNS 0 1 0 1 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0
6 2018 October The Global Financial Environment | Financial Stability Review – October 2018 | RBA Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. 1_frs G1 1 9 6 1_9_6 Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. 118 6 200 19 12.08000 43.48350 6 10 0 6 8 14 14 9 1 2 28 10 3 0 2 1 6 4 1 1 6 2 4 1 0 1 0 0 0 0 0 6.78 11.86 11.86 7.63 0.85 1.69 23.73 8.47 2.54 0.00 1.69 0.85 5.08 3.39 0.85 0.85 5.08 1.69 3.39 0.85 0.00 0.85 0.00 0.00 0.00 0.00 0.00 much JJ much JJ of IN the DT NA NA NA NA NA NA NA NA NA NA 1 1 1 1 0 1 1 0 0
7 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. 1_frs G1 1 8 7 1_8_7 The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. 126 5 237 31 16.43324 22.12843 5 11 11 7 2 18 18 13 1 2 33 9 0 2 5 0 4 4 3 1 6 0 5 0 5 0 4 0 0 0 0 1.48 13.33 13.33 9.63 0.74 1.48 24.44 6.67 0.00 1.48 3.70 0.00 2.96 2.96 2.22 0.74 4.44 0.00 3.70 0.00 3.70 0.00 2.96 0.00 0.00 0.00 0.00 the DT the DT increase NN in IN 1 1 0 0 1 0 0 0 0 0 0 4 1 0 0 0 0 0 0
8 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. 1_frs G1 1 5 8 1_5_8 Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. 176 8 324 33 14.71273 28.76409 11 15 16 8 8 11 27 24 2 1 36 21 0 4 9 1 4 2 1 4 9 4 2 1 4 0 0 0 2 1 0 4.49 6.18 15.17 13.48 1.12 0.56 20.22 11.80 0.00 2.25 5.06 0.56 2.25 1.12 0.56 2.25 5.06 2.25 1.12 0.56 2.25 0.00 0.00 0.00 1.12 0.56 0.00 housing NN housing NN market NN risks NNS 0 1 1 0 0 0 1 1 0 1 1 1 1 0 0 0 4 0 0
9 2019 October Household and Business Finances | Financial Stability Review – October 2019 | RBA Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. 1_frs G1 1 6 9 1_6_9 Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. 211 8 387 53 16.33890 24.89755 12 17 0 9 9 15 28 28 2 3 39 32 1 0 7 2 7 7 0 10 5 11 2 2 0 0 0 0 0 0 1 4.27 7.11 13.27 13.27 0.95 1.42 18.48 15.17 0.47 0.00 3.32 0.95 3.32 3.32 0.00 4.74 2.37 5.21 0.95 0.95 0.00 0.00 0.00 0.00 0.00 0.00 0.47 personal JJ personal JJ debt NN which WDT NA NA NA NA NA NA NA NA NA NA 0 1 3 1 1 0 1 0 1
10 2016 April The Australian Financial System | Financial Stability Review – April 2016 | RBA Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. 1_frs G1 1 2 10 1_2_10 Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. 121 5 228 31 16.08271 22.86043 4 10 19 10 1 19 12 9 0 1 35 5 0 2 4 0 11 6 3 3 5 1 4 0 7 0 2 0 0 0 0 0.77 14.62 9.23 6.92 0.00 0.77 26.92 3.85 0.00 1.54 3.08 0.00 8.46 4.62 2.31 2.31 3.85 0.77 3.08 0.00 5.38 0.00 1.54 0.00 0.00 0.00 0.00 australian NN australian NN banks NNS using VBG NA NA NA NA NA NA NA NA NA NA 0 4 0 1 0 0 0 0 0

4 Extract parse tree features

In this section, we decompose each sentence into a parse tree following the Peenn Treebank syntactic tagset introduced by Taylor, Marcus and Santorini (2003). After that, a number of syntactic structured features embedded in a parse tree are generated, such as:

  • Parse tree types count for a paragraph
  • Parse tree types count for the first sentence of a paragraph
  • Parse tree types count for the last sentence of a paragraph

4.1 Generate parse tree features for each sentence

Decompose each sentence of a given paragraph into a parse tree and then summarise the parse tree features for each paragraph using the openNLP package.A snapshot of the output table is:

# input the pre-defined function. Please be aware that this program requires a large memeory to run, if it doesn't run successfully, please go to the code itself and run it seperately
source("./r_function/tree_parse_feature_extract.R")
##combine the two prase data together
parse_sum <- parse_feature
#A snapshot of the output table is:
parse_sum %>% head() %>% 
  kbl() %>% 
  kable_paper() %>%
  scroll_box(width = "100%", height = "200px")
ADJP ADVP NP PP S SBAR VP WHNP question_index rank
1 1 8 5 2 0 3 0 1_10_1 1
1 0 3 0 2 0 4 0 1_10_1 2
1 0 10 2 6 1 10 0 1_10_1 3
0 1 6 1 3 1 5 1 1_10_1 4
1 1 15 8 2 1 2 0 1_4_2 1
3 0 12 3 4 2 6 0 1_4_2 2

4.2 Extract feature of parse trees for each paragraph

After extracting the parse tree related features for each sentence of sample paragraphs. We constructed three types of parse-related features for each paragraph:

  • Parse tree types count for a paragraph
  • Parse tree types count for the first sentence of a paragraph
  • Parse tree types count for the last sentence of a paragraph

A snapshot of the output table is shown as below:

#create a rank index for each paragraph
parse_sum <- parse_sum %>% group_by (question_index) %>% mutate(rank = c(1:n()))

##create the parse feature for sentence 1
parse_1stsent_feature <- parse_sum %>% dplyr::filter(rank==1)%>% dplyr::select(-rank) 

##rename the columns
colnames(parse_1stsent_feature)[1:ncol(parse_1stsent_feature)-1] <- 
  paste("sent_1st_parse",colnames(parse_1stsent_feature)[1:ncol(parse_1stsent_feature)-1], sep = "_")

library(plyr)

##create the parse feature for the last sentence
parse_lastsent_feature <- parse_sum%>% dplyr::group_by(question_index) %>% dplyr::filter(rank==max(rank))%>% 
  dplyr::select(-rank)
colnames(parse_lastsent_feature)[2:ncol(parse_lastsent_feature)] <- paste("sent_last_parse",colnames(parse_lastsent_feature)[2:ncol(parse_lastsent_feature)], sep = "_")

colnames(parse_lastsent_feature)[1:ncol(parse_lastsent_feature)-1] <- 
  paste("sent_last_parse",colnames(parse_lastsent_feature)[1:ncol(parse_lastsent_feature)-1], sep = "_")

parse_lastsent_feature <- plyr::rename(parse_lastsent_feature, c("sent_last_parse_question_index" = "question_index"))

##create the parse feature for a paragraph (all sentences included)
parse_para_feature <- parse_sum %>% dplyr::select(-rank) %>% dplyr::group_by(question_index) %>%
  dplyr::summarise_each(dplyr::funs(sum)) 

#create the final table for the parse feature
parse_feature <- dplyr::left_join(parse_1stsent_feature, parse_lastsent_feature, by="question_index") %>% 
    dplyr::left_join(.,parse_para_feature,by="question_index") 
#.x is the features for the first sentence, y.is the parse features for the last sentence, and the final section is the features for the whole paragraph


#replace NAs with 0
parse_feature[is.na(parse_feature)] <- 0

#A snapshot of the output table is:
parse_feature %>% head() %>% 
  kbl() %>%
  kable_paper() %>%
  scroll_box(width = "100%", height = "200px")
sent_1st_parse_ADJP sent_1st_parse_ADVP sent_1st_parse_NP sent_1st_parse_PP sent_1st_parse_S sent_1st_parse_SBAR sent_1st_parse_VP sent_1st_parse_WHNP question_index sent_last_parse_ADJP sent_last_parse_sent_last_parse_ADVP sent_last_parse_sent_last_parse_NP sent_last_parse_sent_last_parse_PP sent_last_parse_sent_last_parse_S sent_last_parse_sent_last_parse_SBAR sent_last_parse_sent_last_parse_VP sent_last_parse_sent_last_parse_WHNP ADJP ADVP NP PP S SBAR VP WHNP
1 1 8 5 2 0 3 0 1_10_1 0 1 6 1 3 1 5 1 3 2 27 8 13 2 22 1
1 1 15 8 2 1 2 0 1_4_2 0 0 2 0 2 0 4 0 4 1 29 11 8 3 12 0
2 1 6 3 2 0 5 0 1_9_3 2 1 6 3 2 0 5 0 2 1 6 3 2 0 5 0
0 0 13 5 2 1 4 0 1_3_4 2 1 6 3 5 1 5 0 2 1 27 11 8 2 10 0
0 0 14 5 2 1 3 0 1_3_5 1 1 15 5 3 1 7 0 3 2 37 12 7 3 12 0
3 0 7 4 1 0 3 0 1_9_6 0 0 7 3 4 2 7 1 5 1 44 16 11 3 24 1

5 Save the final dataset

Now, we can create the final text feature table that including all textual features, POS tag feature and syntactic structured features over parse trees. The view of the final output table is shown as below.

text_feature_base <- left_join(text_feature_pos_clue, parse_feature, by="question_index")
#take a look of data

# saveRDS(text_feature_base, "smp_2021_text_feature.rds")
#replace NAs with 0
text_feature_base[is.na(text_feature_base)] <- 0

kbl(text_feature_base) %>%
  kable_paper() %>%
  scroll_box(width = "100%", height = "200px")
X year month issue paragraph source source_group survey_group question_group index.x question_index paragraph_clean word_count_stats sentence_count readability_stats.sylls readability_stats.polys fk_grade_level FRES_score comma_count punc_count digit_count index.y CC DT IN JJ JJR MD NN NNS PRP PRP$ RB RBR TO VB VBD VBG VBN VBP VBZ WDT CD RBS POS RP JJS NNP EX pos_prop_CC pos_prop_DT pos_prop_IN pos_prop_JJ pos_prop_JJR pos_prop_MD pos_prop_NN pos_prop_NNS pos_prop_PRP pos_prop_PRP$ pos_prop_RB pos_prop_RBR pos_prop_TO pos_prop_VB pos_prop_VBD pos_prop_VBG pos_prop_VBN pos_prop_VBP pos_prop_VBZ pos_prop_WDT pos_prop_CD pos_prop_RBS pos_prop_POS pos_prop_RP pos_prop_JJS pos_prop_NNP pos_prop_EX word_choose.x.x word_pos word_choose.x word_pos.word1 word_choose.y word_pos.word2 word_choose.y.y word_pos.word3 sent1st_clue_Attitudinal sent1st_clue_connective sentlast_clue_Attitudinal sentlast_clue_connective sentlast_clue_Contrast sentlast_clue_summary sentmiddle_clue_Attitudinal sentmiddle_clue_connective sentmiddle_clue_detail sentmiddle_clue_inference sent_1st_word_CC sent_1st_word_DT sent_1st_word_IN sent_1st_word_JJ sent_1st_word_JJR sent_1st_word_JJS sent_1st_word_NN sent_1st_word_PRP sent_1st_word_RB sent_1st_parse_ADJP sent_1st_parse_ADVP sent_1st_parse_NP sent_1st_parse_PP sent_1st_parse_S sent_1st_parse_SBAR sent_1st_parse_VP sent_1st_parse_WHNP sent_last_parse_ADJP sent_last_parse_sent_last_parse_ADVP sent_last_parse_sent_last_parse_NP sent_last_parse_sent_last_parse_PP sent_last_parse_sent_last_parse_S sent_last_parse_sent_last_parse_SBAR sent_last_parse_sent_last_parse_VP sent_last_parse_sent_last_parse_WHNP ADJP ADVP NP PP S SBAR VP WHNP
1 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. 1_frs G1 1 10 1 1_10_1 Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. 92 4 163 17 14.28652 33.60087 3 9 0 1 5 6 8 13 1 3 14 12 2 1 5 1 4 6 2 1 2 3 1 1 0 0 0 0 0 0 0 5.49 6.59 8.79 14.29 1.10 3.30 15.38 13.19 2.20 1.10 5.49 1.10 4.40 6.59 2.20 1.10 2.20 3.30 1.10 1.10 0.00 0.00 0.00 0.00 0.00 0.00 0.00 along IN along IN with IN the DT 1 1 0 1 0 0 0 0 0 0 0 1 2 0 0 0 0 1 0 1 1 8 5 2 0 3 0 0 1 6 1 3 1 5 1 3 2 27 8 13 2 22 1
2 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. 1_frs G1 1 4 2 1_4_2 Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. 82 3 150 15 16.65537 24.33557 3 3 5 2 2 6 13 11 1 2 16 12 1 0 5 1 2 3 2 0 2 2 1 0 2 0 0 0 0 0 0 2.38 7.14 15.48 13.10 1.19 2.38 19.05 14.29 1.19 0.00 5.95 1.19 2.38 3.57 2.38 0.00 2.38 2.38 1.19 0.00 2.38 0.00 0.00 0.00 0.00 0.00 0.00 total JJ total JJ superannuation NN assets NNS 2 0 0 0 0 1 0 0 0 0 0 0 1 1 0 0 1 0 0 1 1 15 8 2 1 2 0 0 0 2 0 2 0 4 0 4 1 29 11 8 3 12 0
3 2017 October The Global Financial Environment | Financial Stability Review – October 2017 | RBA Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. 1_frs G1 1 9 3 1_9_3 Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. 25 1 54 8 19.64800 -1.27600 1 1 0 3 1 1 3 4 1 0 1 4 0 0 2 0 1 2 0 2 1 1 0 0 0 1 0 0 0 0 0 4.00 4.00 12.00 16.00 4.00 0.00 4.00 16.00 0.00 0.00 8.00 0.00 4.00 8.00 0.00 8.00 4.00 4.00 0.00 0.00 0.00 4.00 0.00 0.00 0.00 0.00 0.00 despite IN despite IN challenging VBG economic JJ 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 1 6 3 2 0 5 0 2 1 6 3 2 0 5 0 2 1 6 3 2 0 5 0
4 2016 October The Global Financial Environment | Financial Stability Review – October 2016 | RBA With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. 1_frs G1 1 3 4 1_3_4 With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. 73 3 143 20 17.01507 16.41338 5 4 0 4 4 5 10 12 0 0 14 11 0 0 2 0 3 1 0 3 2 3 2 0 0 0 1 1 0 0 0 5.41 6.76 13.51 16.22 0.00 0.00 18.92 14.86 0.00 0.00 2.70 0.00 4.05 1.35 0.00 4.05 2.70 4.05 2.70 0.00 0.00 0.00 1.35 1.35 0.00 0.00 0.00 with IN with IN the DT increasing VBG 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 1 0 0 0 0 13 5 2 1 4 0 2 1 6 3 5 1 5 0 2 1 27 11 8 2 10 0
5 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. 1_frs G1 1 3 5 1_3_5 If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. 89 3 163 21 17.59124 21.78176 3 6 0 5 5 3 15 16 0 2 15 11 2 0 5 0 2 2 0 2 1 5 1 0 0 0 1 1 0 0 0 5.62 3.37 16.85 17.98 0.00 2.25 16.85 12.36 2.25 0.00 5.62 0.00 2.25 2.25 0.00 2.25 1.12 5.62 1.12 0.00 0.00 0.00 1.12 1.12 0.00 0.00 0.00 if IN if IN financial JJ strains NNS 0 1 0 1 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 0 14 5 2 1 3 0 1 1 15 5 3 1 7 0 3 2 37 12 7 3 12 0
6 2018 October The Global Financial Environment | Financial Stability Review – October 2018 | RBA Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. 1_frs G1 1 9 6 1_9_6 Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. 118 6 200 19 12.08000 43.48350 6 10 0 6 8 14 14 9 1 2 28 10 3 0 2 1 6 4 1 1 6 2 4 1 0 1 0 0 0 0 0 6.78 11.86 11.86 7.63 0.85 1.69 23.73 8.47 2.54 0.00 1.69 0.85 5.08 3.39 0.85 0.85 5.08 1.69 3.39 0.85 0.00 0.85 0.00 0.00 0.00 0.00 0.00 much JJ much JJ of IN the DT 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 1 1 0 0 3 0 7 4 1 0 3 0 0 0 7 3 4 2 7 1 5 1 44 16 11 3 24 1
7 2016 October The Australian Financial System | Financial Stability Review – October 2016 | RBA The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. 1_frs G1 1 8 7 1_8_7 The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. 126 5 237 31 16.43324 22.12843 5 11 11 7 2 18 18 13 1 2 33 9 0 2 5 0 4 4 3 1 6 0 5 0 5 0 4 0 0 0 0 1.48 13.33 13.33 9.63 0.74 1.48 24.44 6.67 0.00 1.48 3.70 0.00 2.96 2.96 2.22 0.74 4.44 0.00 3.70 0.00 3.70 0.00 2.96 0.00 0.00 0.00 0.00 the DT the DT increase NN in IN 1 1 0 0 1 0 0 0 0 0 0 4 1 0 0 0 0 0 0 1 1 8 5 2 1 4 0 0 0 6 3 1 0 1 0 3 4 38 16 11 5 21 0
8 2016 April The Global Financial Environment | Financial Stability Review – April 2016 | RBA Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. 1_frs G1 1 5 8 1_5_8 Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. 176 8 324 33 14.71273 28.76409 11 15 16 8 8 11 27 24 2 1 36 21 0 4 9 1 4 2 1 4 9 4 2 1 4 0 0 0 2 1 0 4.49 6.18 15.17 13.48 1.12 0.56 20.22 11.80 0.00 2.25 5.06 0.56 2.25 1.12 0.56 2.25 5.06 2.25 1.12 0.56 2.25 0.00 0.00 0.00 1.12 0.56 0.00 housing NN housing NN market NN risks NNS 0 1 1 0 0 0 1 1 0 1 1 1 1 0 0 0 4 0 0 1 1 4 1 1 0 1 0 1 1 19 9 1 0 2 0 2 9 64 29 11 1 20 1
9 2019 October Household and Business Finances | Financial Stability Review – October 2019 | RBA Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. 1_frs G1 1 6 9 1_6_9 Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. 211 8 387 53 16.33890 24.89755 12 17 0 9 9 15 28 28 2 3 39 32 1 0 7 2 7 7 0 10 5 11 2 2 0 0 0 0 0 0 1 4.27 7.11 13.27 13.27 0.95 1.42 18.48 15.17 0.47 0.00 3.32 0.95 3.32 3.32 0.00 4.74 2.37 5.21 0.95 0.95 0.00 0.00 0.00 0.00 0.00 0.00 0.47 personal JJ personal JJ debt NN which WDT 0 0 0 0 0 0 0 0 0 0 0 1 3 1 1 0 1 0 1 0 0 14 3 1 1 1 1 0 1 11 5 1 0 2 0 3 4 82 29 24 6 39 2
10 2016 April The Australian Financial System | Financial Stability Review – April 2016 | RBA Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. 1_frs G1 1 2 10 1_2_10 Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. 121 5 228 31 16.08271 22.86043 4 10 19 10 1 19 12 9 0 1 35 5 0 2 4 0 11 6 3 3 5 1 4 0 7 0 2 0 0 0 0 0.77 14.62 9.23 6.92 0.00 0.77 26.92 3.85 0.00 1.54 3.08 0.00 8.46 4.62 2.31 2.31 3.85 0.77 3.08 0.00 5.38 0.00 1.54 0.00 0.00 0.00 0.00 australian NN australian NN banks NNS using VBG 0 0 0 0 0 0 0 0 0 0 0 4 0 1 0 0 0 0 0 0 0 6 2 2 0 6 0 0 2 9 4 2 0 4 0 1 4 40 15 13 2 28 0

6 Session information

The session information for this program is:

sessionInfo()
## R version 4.0.3 (2020-10-10)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 17763)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=English_Australia.1252  LC_CTYPE=English_Australia.1252   
## [3] LC_MONETARY=English_Australia.1252 LC_NUMERIC=C                      
## [5] LC_TIME=English_Australia.1252    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] plyr_1.8.6             openNLP_0.2-7          sylcount_0.2-2        
##  [4] rvest_1.0.0            xml2_1.3.2             forcats_0.5.1         
##  [7] purrr_0.3.4            readr_1.4.0            tidyverse_1.3.1       
## [10] quanteda_3.0.0         textclean_0.9.3        kableExtra_1.3.4      
## [13] tidyr_1.1.3            MASS_7.3-53            data.table_1.14.0     
## [16] lubridate_1.7.10       tibble_3.1.1           zoo_1.8-9             
## [19] reshape2_1.4.4         stringi_1.5.3          reshape_0.8.8         
## [22] qdap_2.4.3             qdapTools_1.3.5        qdapRegex_0.7.2       
## [25] qdapDictionaries_1.0.7 stargazer_5.2.2        e1071_1.7-6           
## [28] stringr_1.4.0          tm_0.7-8               NLP_0.2-1             
## [31] wordcloud_2.6          RColorBrewer_1.1-2     ggplot2_3.3.3         
## [34] tidytext_0.3.1         dplyr_1.0.6           
## 
## loaded via a namespace (and not attached):
##  [1] fs_1.5.0            bitops_1.0-7        webshot_0.5.2      
##  [4] httr_1.4.2          SnowballC_0.7.0     tools_4.0.3        
##  [7] backports_1.2.1     utf8_1.2.1          R6_2.5.0           
## [10] DBI_1.1.1           colorspace_2.0-1    openNLPdata_1.5.3-4
## [13] withr_2.4.2         tidyselect_1.1.1    gridExtra_2.3      
## [16] compiler_4.0.3      chron_2.3-56        cli_2.5.0          
## [19] slam_0.1-48         scales_1.1.1        proxy_0.4-25       
## [22] systemfonts_1.0.1   digest_0.6.27       rmarkdown_2.8      
## [25] svglite_2.0.0       pkgconfig_2.0.3     htmltools_0.5.1.1  
## [28] plotrix_3.8-1       highr_0.9           dbplyr_2.1.1       
## [31] readxl_1.3.1        rlang_0.4.11        rstudioapi_0.13    
## [34] generics_0.1.0      jsonlite_1.7.2      zip_2.1.1          
## [37] tokenizers_0.2.1    RCurl_1.98-1.3      magrittr_2.0.1     
## [40] Matrix_1.2-18       Rcpp_1.0.6          munsell_0.5.0      
## [43] fansi_0.4.2         lifecycle_1.0.0     yaml_2.2.1         
## [46] grid_4.0.3          parallel_4.0.3      gender_0.5.4       
## [49] crayon_1.4.1        lattice_0.20-41     haven_2.4.1        
## [52] hms_1.0.0           knitr_1.33          venneuler_1.1-0    
## [55] pillar_1.6.0        igraph_1.2.6        stopwords_2.2      
## [58] fastmatch_1.1-0     reprex_2.0.0        XML_3.99-0.6       
## [61] glue_1.4.2          evaluate_0.14       modelr_0.1.8       
## [64] RcppParallel_5.1.4  vctrs_0.3.8         cellranger_1.1.0   
## [67] gtable_0.3.0        assertthat_0.2.1    xfun_0.22          
## [70] openxlsx_4.2.3      broom_0.7.6         janeaustenr_0.1.5  
## [73] class_7.3-17        viridisLite_0.4.0   rJava_1.0-4        
## [76] ellipsis_0.3.2