This code shows the process of how we extract text-related features for sample paragraphs. Those features include text statistics, part-of-speech (POS) tags and syntactic structured features over parse trees (parse tree features). It is organised as follows. Section 2 loads libraries and sample paragraphs; section 3 is the process of extracting all textual features and POS tag features and Section 4 shows the process of how to extract parse tree features for each paragraph. Section 5 generates the final output table and the last session provides session information.
Loading required packages. You may need to install some packages first if they are failed to be loaded.
load.lib<-c("xlsx", "dplyr","tidytext","ggplot2","ggthemes","wordcloud",
"tm","stringr","e1071","ldatuning","pander","dplyr","pdftools",
"stargazer","qdap","reshape","sqldf","stringi","stringr","reshape2",
"readtext","zoo", "tibble", "lubridate", "data.table", "texreg",
"MASS", "AER", "pscl","tidyr", "kableExtra", "textclean", "quanteda",
"tidytext", "tidyverse")
sapply(load.lib,require,character=TRUE)
mutate <- dplyr::mutate
select <- dplyr::select
count <- dplyr::count
summarise <- dplyr::summarise
rename <- plyr::renameLoading pre-defined functions that will be used later in this code.
source("./r_function/text_stats.R")
source("./r_function/POS_tag_function.R")
source("./r_function/function_sentence_feature.R")Input sample paragraphs.
#set up the folder to read sample paragraphs
text_file_path <- "../Survey Data/3_survey_group"
#read 5 surveys and combine them as one documents
file_list = list.files(text_file_path, pattern="*.csv")
file_MergedData <- do.call(rbind,lapply(paste(text_file_path, file_list, sep = "/"), read.csv))
#rename the column name and clean
file_MergedData <- plyr::rename(file_MergedData, c("para" = "paragraph"))Clean text by replacing unrecognised characteristics such as “`”. This is an important step as those unrecognised characteristic will impact the accuracy of decomposing a paragraph into sentences. The distribution sample paragraphs by source is shown in the table below:
#replace non-English symbols in the text
file_MergedData$paragraph <- str_replace_all(file_MergedData$paragraph, "`","'")
file_MergedData$paragraph <- str_replace_all(file_MergedData$paragraph, "'","'")
file_MergedData$paragraph <- str_replace_all(file_MergedData$paragraph, "'","'")
# file_MergedData$paragraph <- str_replace_all(file_MergedData$paragraph, "??","'''")
#remove brackets and the contents between it
file_MergedData$paragraph <- bracketX(file_MergedData$paragraph)#remove the brackets and its contents in it
#remove extra white spaces
file_MergedData$paragraph <- rm_white_endmark(file_MergedData$paragraph)
file_MergedData$paragraph <- rm_white_lead_trail(file_MergedData$paragraph)
file_MergedData$paragraph <- rm_white_multiple(file_MergedData$paragraph)
file_MergedData$paragraph <- rm_white_punctuation(file_MergedData$paragraph)
file_MergedData$paragraph <- rm_white_comma(file_MergedData$paragraph)
#create an index column for recording each row
file_MergedData$index <- 1:nrow(file_MergedData)
# file_MergedData %>% top_n(.,5) %>% kbl()
text_source_tb <-
file_MergedData %>% group_by(source) %>% summarise(n=n())%>% mutate(rel.freq = paste0(round(100 * n/sum(n), 0), "%"))
text_source_tb %>%
kbl(caption = "Composition of sample paragraph") %>%
kable_classic(full_width = F, html_font = "Cambria")| source | n | rel.freq |
|---|---|---|
| 1_frs | 50 | 5% |
| 10_economist | 200 | 20% |
| 11_grattan | 100 | 10% |
| 2_bulletin | 100 | 10% |
| 3_rba_speeches | 100 | 10% |
| 4_smp_intro_2006_2019 | 100 | 10% |
| 5_smp_main | 50 | 5% |
| 6_smp_boxes_06_19 | 100 | 10% |
| 7_boe_main | 50 | 5% |
| 8_boe_ir_intro | 50 | 5% |
| 9_boe_speeches | 100 | 10% |
In this section, we generate the text related features all textual features, readability, argument features and syntactic features excluding structured synthetics feature over parse trees.
Create a unique index number to record each paragraph. This variable will be used as a key to join other tables. The NLP package requires a large computing power, so running this code using the whole sample size may fail. Given the main purpose of this code is to show the process of generating text-related features, we limit the sample size to 10 paragraphs to ensure the code run smoothly.
## convert all factor variables to characters
file_MergedData %>% mutate_if(is.factor, as.character) -> file_MergedData
##create an unique index column
file_MergedData$question_index <- paste(file_MergedData$survey_group, file_MergedData$question_group,
file_MergedData$index, sep = "_")
#limited to 10 paragraphs to allow the following programs run smoothly
file_MergedData <- file_MergedData[1:10,]This code extracts textual features and readability features including:
A snapshot of the output is shown as below.
## create the feature list table
survey_feature1 <- para_stats_function(file_MergedData)
#A snapshot of the data is:
kbl(survey_feature1) %>%
kable_paper() %>%
scroll_box(width = "100%", height = "200px")| X | year | month | issue | paragraph | source | source_group | survey_group | question_group | index | question_index | paragraph_clean | word_count_stats | sentence_count | readability_stats.sylls | readability_stats.polys | fk_grade_level | FRES_score | comma_count | punc_count | digit_count |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. | 1_frs | G1 | 1 | 10 | 1 | 1_10_1 | Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. | 92 | 4 | 163 | 17 | 14.28652 | 33.60087 | 3 | 9 | 0 |
| 2 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. | 1_frs | G1 | 1 | 4 | 2 | 1_4_2 | Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. | 82 | 3 | 150 | 15 | 16.65537 | 24.33557 | 3 | 3 | 5 |
| 3 | 2017 | October | The Global Financial Environment | Financial Stability Review – October 2017 | RBA | Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. | 1_frs | G1 | 1 | 9 | 3 | 1_9_3 | Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. | 25 | 1 | 54 | 8 | 19.64800 | -1.27600 | 1 | 1 | 0 |
| 4 | 2016 | October | The Global Financial Environment | Financial Stability Review – October 2016 | RBA | With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. | 1_frs | G1 | 1 | 3 | 4 | 1_3_4 | With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. | 73 | 3 | 143 | 20 | 17.01507 | 16.41338 | 5 | 4 | 0 |
| 5 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. | 1_frs | G1 | 1 | 3 | 5 | 1_3_5 | If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. | 89 | 3 | 163 | 21 | 17.59124 | 21.78176 | 3 | 6 | 0 |
| 6 | 2018 | October | The Global Financial Environment | Financial Stability Review – October 2018 | RBA | Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. | 1_frs | G1 | 1 | 9 | 6 | 1_9_6 | Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. | 118 | 6 | 200 | 19 | 12.08000 | 43.48350 | 6 | 10 | 0 |
| 7 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. | 1_frs | G1 | 1 | 8 | 7 | 1_8_7 | The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. | 126 | 5 | 237 | 31 | 16.43324 | 22.12843 | 5 | 11 | 11 |
| 8 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. | 1_frs | G1 | 1 | 5 | 8 | 1_5_8 | Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. | 176 | 8 | 324 | 33 | 14.71273 | 28.76409 | 11 | 15 | 16 |
| 9 | 2019 | October | Household and Business Finances | Financial Stability Review – October 2019 | RBA | Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. | 1_frs | G1 | 1 | 6 | 9 | 1_6_9 | Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. | 211 | 8 | 387 | 53 | 16.33890 | 24.89755 | 12 | 17 | 0 |
| 10 | 2016 | April | The Australian Financial System | Financial Stability Review – April 2016 | RBA | Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. | 1_frs | G1 | 1 | 2 | 10 | 1_2_10 | Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. | 121 | 5 | 228 | 31 | 16.08271 | 22.86043 | 4 | 10 | 19 |
This section extracts POS tag and argument features for a given pargaph and its sentences.
Generate POS features for a given paragraph, including: * POS count * POS ratio A snapshot of the output is shown as below.
survey_feature2 <- word_pos_prop_function(survey_feature1) #calculate the POS taggers for each paragraph
# head(survey_feature2)
## calculate the proportions of each tag in the paragraph (pos tag count / total words)
para_pos_prop <- round(survey_feature2[-1:-2]/rowSums(survey_feature2[-1:-2]),4)*100
colnames(para_pos_prop) <- paste("pos_prop",colnames(para_pos_prop), sep = "_")
survey_feature2 <- cbind(survey_feature2,para_pos_prop)
survey_feature_part2 <- left_join(survey_feature1,survey_feature2, by="question_index")
#A snapshot of the output from this step is:
kbl(survey_feature_part2) %>%
kable_paper() %>%
scroll_box(width = "100%", height = "200px")| X | year | month | issue | paragraph | source | source_group | survey_group | question_group | index.x | question_index | paragraph_clean | word_count_stats | sentence_count | readability_stats.sylls | readability_stats.polys | fk_grade_level | FRES_score | comma_count | punc_count | digit_count | index.y | CC | DT | IN | JJ | JJR | MD | NN | NNS | PRP | PRP$ | RB | RBR | TO | VB | VBD | VBG | VBN | VBP | VBZ | WDT | CD | RBS | POS | RP | JJS | NNP | EX | pos_prop_CC | pos_prop_DT | pos_prop_IN | pos_prop_JJ | pos_prop_JJR | pos_prop_MD | pos_prop_NN | pos_prop_NNS | pos_prop_PRP | pos_prop_PRP$ | pos_prop_RB | pos_prop_RBR | pos_prop_TO | pos_prop_VB | pos_prop_VBD | pos_prop_VBG | pos_prop_VBN | pos_prop_VBP | pos_prop_VBZ | pos_prop_WDT | pos_prop_CD | pos_prop_RBS | pos_prop_POS | pos_prop_RP | pos_prop_JJS | pos_prop_NNP | pos_prop_EX |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. | 1_frs | G1 | 1 | 10 | 1 | 1_10_1 | Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. | 92 | 4 | 163 | 17 | 14.28652 | 33.60087 | 3 | 9 | 0 | 1 | 5 | 6 | 8 | 13 | 1 | 3 | 14 | 12 | 2 | 1 | 5 | 1 | 4 | 6 | 2 | 1 | 2 | 3 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 5.49 | 6.59 | 8.79 | 14.29 | 1.10 | 3.30 | 15.38 | 13.19 | 2.20 | 1.10 | 5.49 | 1.10 | 4.40 | 6.59 | 2.20 | 1.10 | 2.20 | 3.30 | 1.10 | 1.10 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| 2 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. | 1_frs | G1 | 1 | 4 | 2 | 1_4_2 | Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. | 82 | 3 | 150 | 15 | 16.65537 | 24.33557 | 3 | 3 | 5 | 2 | 2 | 6 | 13 | 11 | 1 | 2 | 16 | 12 | 1 | 0 | 5 | 1 | 2 | 3 | 2 | 0 | 2 | 2 | 1 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 2.38 | 7.14 | 15.48 | 13.10 | 1.19 | 2.38 | 19.05 | 14.29 | 1.19 | 0.00 | 5.95 | 1.19 | 2.38 | 3.57 | 2.38 | 0.00 | 2.38 | 2.38 | 1.19 | 0.00 | 2.38 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| 3 | 2017 | October | The Global Financial Environment | Financial Stability Review – October 2017 | RBA | Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. | 1_frs | G1 | 1 | 9 | 3 | 1_9_3 | Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. | 25 | 1 | 54 | 8 | 19.64800 | -1.27600 | 1 | 1 | 0 | 3 | 1 | 1 | 3 | 4 | 1 | 0 | 1 | 4 | 0 | 0 | 2 | 0 | 1 | 2 | 0 | 2 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 4.00 | 4.00 | 12.00 | 16.00 | 4.00 | 0.00 | 4.00 | 16.00 | 0.00 | 0.00 | 8.00 | 0.00 | 4.00 | 8.00 | 0.00 | 8.00 | 4.00 | 4.00 | 0.00 | 0.00 | 0.00 | 4.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| 4 | 2016 | October | The Global Financial Environment | Financial Stability Review – October 2016 | RBA | With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. | 1_frs | G1 | 1 | 3 | 4 | 1_3_4 | With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. | 73 | 3 | 143 | 20 | 17.01507 | 16.41338 | 5 | 4 | 0 | 4 | 4 | 5 | 10 | 12 | 0 | 0 | 14 | 11 | 0 | 0 | 2 | 0 | 3 | 1 | 0 | 3 | 2 | 3 | 2 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 5.41 | 6.76 | 13.51 | 16.22 | 0.00 | 0.00 | 18.92 | 14.86 | 0.00 | 0.00 | 2.70 | 0.00 | 4.05 | 1.35 | 0.00 | 4.05 | 2.70 | 4.05 | 2.70 | 0.00 | 0.00 | 0.00 | 1.35 | 1.35 | 0.00 | 0.00 | 0.00 |
| 5 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. | 1_frs | G1 | 1 | 3 | 5 | 1_3_5 | If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. | 89 | 3 | 163 | 21 | 17.59124 | 21.78176 | 3 | 6 | 0 | 5 | 5 | 3 | 15 | 16 | 0 | 2 | 15 | 11 | 2 | 0 | 5 | 0 | 2 | 2 | 0 | 2 | 1 | 5 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 5.62 | 3.37 | 16.85 | 17.98 | 0.00 | 2.25 | 16.85 | 12.36 | 2.25 | 0.00 | 5.62 | 0.00 | 2.25 | 2.25 | 0.00 | 2.25 | 1.12 | 5.62 | 1.12 | 0.00 | 0.00 | 0.00 | 1.12 | 1.12 | 0.00 | 0.00 | 0.00 |
| 6 | 2018 | October | The Global Financial Environment | Financial Stability Review – October 2018 | RBA | Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. | 1_frs | G1 | 1 | 9 | 6 | 1_9_6 | Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. | 118 | 6 | 200 | 19 | 12.08000 | 43.48350 | 6 | 10 | 0 | 6 | 8 | 14 | 14 | 9 | 1 | 2 | 28 | 10 | 3 | 0 | 2 | 1 | 6 | 4 | 1 | 1 | 6 | 2 | 4 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 6.78 | 11.86 | 11.86 | 7.63 | 0.85 | 1.69 | 23.73 | 8.47 | 2.54 | 0.00 | 1.69 | 0.85 | 5.08 | 3.39 | 0.85 | 0.85 | 5.08 | 1.69 | 3.39 | 0.85 | 0.00 | 0.85 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| 7 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. | 1_frs | G1 | 1 | 8 | 7 | 1_8_7 | The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. | 126 | 5 | 237 | 31 | 16.43324 | 22.12843 | 5 | 11 | 11 | 7 | 2 | 18 | 18 | 13 | 1 | 2 | 33 | 9 | 0 | 2 | 5 | 0 | 4 | 4 | 3 | 1 | 6 | 0 | 5 | 0 | 5 | 0 | 4 | 0 | 0 | 0 | 0 | 1.48 | 13.33 | 13.33 | 9.63 | 0.74 | 1.48 | 24.44 | 6.67 | 0.00 | 1.48 | 3.70 | 0.00 | 2.96 | 2.96 | 2.22 | 0.74 | 4.44 | 0.00 | 3.70 | 0.00 | 3.70 | 0.00 | 2.96 | 0.00 | 0.00 | 0.00 | 0.00 |
| 8 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. | 1_frs | G1 | 1 | 5 | 8 | 1_5_8 | Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. | 176 | 8 | 324 | 33 | 14.71273 | 28.76409 | 11 | 15 | 16 | 8 | 8 | 11 | 27 | 24 | 2 | 1 | 36 | 21 | 0 | 4 | 9 | 1 | 4 | 2 | 1 | 4 | 9 | 4 | 2 | 1 | 4 | 0 | 0 | 0 | 2 | 1 | 0 | 4.49 | 6.18 | 15.17 | 13.48 | 1.12 | 0.56 | 20.22 | 11.80 | 0.00 | 2.25 | 5.06 | 0.56 | 2.25 | 1.12 | 0.56 | 2.25 | 5.06 | 2.25 | 1.12 | 0.56 | 2.25 | 0.00 | 0.00 | 0.00 | 1.12 | 0.56 | 0.00 |
| 9 | 2019 | October | Household and Business Finances | Financial Stability Review – October 2019 | RBA | Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. | 1_frs | G1 | 1 | 6 | 9 | 1_6_9 | Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. | 211 | 8 | 387 | 53 | 16.33890 | 24.89755 | 12 | 17 | 0 | 9 | 9 | 15 | 28 | 28 | 2 | 3 | 39 | 32 | 1 | 0 | 7 | 2 | 7 | 7 | 0 | 10 | 5 | 11 | 2 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 4.27 | 7.11 | 13.27 | 13.27 | 0.95 | 1.42 | 18.48 | 15.17 | 0.47 | 0.00 | 3.32 | 0.95 | 3.32 | 3.32 | 0.00 | 4.74 | 2.37 | 5.21 | 0.95 | 0.95 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.47 |
| 10 | 2016 | April | The Australian Financial System | Financial Stability Review – April 2016 | RBA | Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. | 1_frs | G1 | 1 | 2 | 10 | 1_2_10 | Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. | 121 | 5 | 228 | 31 | 16.08271 | 22.86043 | 4 | 10 | 19 | 10 | 1 | 19 | 12 | 9 | 0 | 1 | 35 | 5 | 0 | 2 | 4 | 0 | 11 | 6 | 3 | 3 | 5 | 1 | 4 | 0 | 7 | 0 | 2 | 0 | 0 | 0 | 0 | 0.77 | 14.62 | 9.23 | 6.92 | 0.00 | 0.77 | 26.92 | 3.85 | 0.00 | 1.54 | 3.08 | 0.00 | 8.46 | 4.62 | 2.31 | 2.31 | 3.85 | 0.77 | 3.08 | 0.00 | 5.38 | 0.00 | 1.54 | 0.00 | 0.00 | 0.00 | 0.00 |
Decompose each paragraph into sentences to generate sentence-related POS features, including:
Extract sentences for a given paragraph using unnest_tokens() function, and generaing three output table:
## untoken paragraphs to sentences and rank each sentence
text_sentence <- file_MergedData %>%
unnest_tokens(sentence, paragraph, token = "sentences") %>%
group_by(question_index) %>%
mutate(rank = c(1:n())) #tokenize to sentence and add a column called 'rank' to extract the first sentence of each group.
## sentence feature 1: check if the first sentence include words "table", "figure", "graph" or include numbers
first_sentence <- text_sentence %>%
select(question_index, sentence, rank) %>%
filter(rank == 1) %>%
mutate(tf_index = ifelse(str_detect(sentence, "\\table\\b|graph|chart|figure"),1,0),
number_index = ifelse(str_detect(sentence, "[0123456789]"),1,0)) #check if the first sentence include table, graph, chart or figure - those would indicate that the dicussion of the whole sentence will be based on facts;
first_sentence$sentence <- gsub("&", "and", first_sentence$sentence) #replace & with "and"
### extract the last sentence
last_sentence <- text_sentence %>%
group_by(question_index) %>%
filter(rank == max(rank)) %>%
select(question_index, sentence, rank)
last_sentence$sentence <- gsub("&", "and", last_sentence$sentence) #replace & with "and"
#A snapshot of the output table of the text_setence table
kbl(text_sentence) %>%
kable_paper() %>%
scroll_box(width = "100%", height = "200px")| X | year | month | issue | source | source_group | survey_group | question_group | index | question_index | sentence | rank |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 10 | 1 | 1_10_1 | along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. | 1 |
| 1 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 10 | 1 | 1_10_1 | this has increased their liquidity risks and made them even more interconnected and systemic. | 2 |
| 1 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 10 | 1 | 1_10_1 | if corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. | 3 |
| 1 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 10 | 1 | 1_10_1 | it could also transmit distress to other institutions that investors consider to have a similar vulnerability. | 4 |
| 2 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | 1_frs | G1 | 1 | 4 | 2 | 1_4_2 | total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. | 1 |
| 2 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | 1_frs | G1 | 1 | 4 | 2 | 1_4_2 | while net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. | 2 |
| 2 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | 1_frs | G1 | 1 | 4 | 2 | 1_4_2 | superannuation funds will therefore need to consider the associated liquidity implications. | 3 |
| 3 | 2017 | October | The Global Financial Environment | Financial Stability Review – October 2017 | RBA | 1_frs | G1 | 1 | 9 | 3 | 1_9_3 | despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. | 1 |
| 4 | 2016 | October | The Global Financial Environment | Financial Stability Review – October 2016 | RBA | 1_frs | G1 | 1 | 3 | 4 | 1_3_4 | with the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. | 1 |
| 4 | 2016 | October | The Global Financial Environment | Financial Stability Review – October 2016 | RBA | 1_frs | G1 | 1 | 3 | 4 | 1_3_4 | as for china, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. | 2 |
| 4 | 2016 | October | The Global Financial Environment | Financial Stability Review – October 2016 | RBA | 1_frs | G1 | 1 | 3 | 4 | 1_3_4 | lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. | 3 |
| 5 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 3 | 5 | 1_3_5 | if financial strains that threaten growth in china emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. | 1 |
| 5 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 3 | 5 | 1_3_5 | direct financial linkages between china and other economies are small in aggregate because china’s capital account is still relatively closed. | 2 |
| 5 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 3 | 5 | 1_3_5 | but these linkages have grown – both in terms of foreign bank lending to china and chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. | 3 |
| 6 | 2018 | October | The Global Financial Environment | Financial Stability Review – October 2018 | RBA | 1_frs | G1 | 1 | 9 | 6 | 1_9_6 | much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent nbfis. | 1 |
| 6 | 2018 | October | The Global Financial Environment | Financial Stability Review – October 2018 | RBA | 1_frs | G1 | 1 | 9 | 6 | 1_9_6 | most of this lending is ultimately funded by the banking sector. | 2 |
| 6 | 2018 | October | The Global Financial Environment | Financial Stability Review – October 2018 | RBA | 1_frs | G1 | 1 | 9 | 6 | 1_9_6 | while this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. | 3 |
| 6 | 2018 | October | The Global Financial Environment | Financial Stability Review – October 2018 | RBA | 1_frs | G1 | 1 | 9 | 6 | 1_9_6 | the riskier nature of the lending, and the obscure and complex interconnections between nbfis and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. | 4 |
| 6 | 2018 | October | The Global Financial Environment | Financial Stability Review – October 2018 | RBA | 1_frs | G1 | 1 | 9 | 6 | 1_9_6 | loan losses and defaults have been modest to date. | 5 |
| 6 | 2018 | October | The Global Financial Environment | Financial Stability Review – October 2018 | RBA | 1_frs | G1 | 1 | 9 | 6 | 1_9_6 | but if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. | 6 |
| 7 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | 1_frs | G1 | 1 | 8 | 7 | 1_8_7 | the increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. | 1 |
| 7 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | 1_frs | G1 | 1 | 8 | 7 | 1_8_7 | the leverage ratio is a non-risk based measure of a bank’s tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. | 2 |
| 7 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | 1_frs | G1 | 1 | 8 | 7 | 1_8_7 | the leverage ratio framework is yet to be finalised internationally, although the basel committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from january 2018. | 3 |
| 7 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | 1_frs | G1 | 1 | 8 | 7 | 1_8_7 | each of the major banks’ leverage ratios was around 5 per cent at june 2016, well above that minimum. | 4 |
| 7 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | 1_frs | G1 | 1 | 8 | 7 | 1_8_7 | at this level, the major australian banks’ leverage ratio sits around the median of international banks. | 5 |
| 8 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 5 | 8 | 1_5_8 | housing market risks are also present in some emerging market and asian economies. | 1 |
| 8 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 5 | 8 | 1_5_8 | this reflects large increases in residential property prices over recent years – including in hong kong, brazil, malaysia, taiwan and turkey – alongside increased household indebtedness. | 2 |
| 8 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 5 | 8 | 1_5_8 | price growth has moderated more recently and prices have fallen in some economies, including brazil, russia and taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. | 3 |
| 8 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 5 | 8 | 1_5_8 | housing prices in hong kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. | 4 |
| 8 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 5 | 8 | 1_5_8 | but prices have fallen recently amid concerns about economic conditions in china and slower credit growth. | 5 |
| 8 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 5 | 8 | 1_5_8 | housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. | 6 |
| 8 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 5 | 8 | 1_5_8 | despite the slowdown in the housing market, the hong kong monetary authority imposed a countercyclical capital buffer of 0.625 per cent in january 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-gdp and housing prices-to-rents relative to their long-run trends. | 7 |
| 9 | 2019 | October | Household and Business Finances | Financial Stability Review – October 2019 | RBA | 1_frs | G1 | 1 | 6 | 9 | 1_6_9 | personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. | 1 |
| 9 | 2019 | October | Household and Business Finances | Financial Stability Review – October 2019 | RBA | 1_frs | G1 | 1 | 6 | 9 | 1_6_9 | in recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. | 2 |
| 9 | 2019 | October | Household and Business Finances | Financial Stability Review – October 2019 | RBA | 1_frs | G1 | 1 | 6 | 9 | 1_6_9 | in part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. | 3 |
| 9 | 2019 | October | Household and Business Finances | Financial Stability Review – October 2019 | RBA | 1_frs | G1 | 1 | 6 | 9 | 1_6_9 | more recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. | 4 |
| 9 | 2019 | October | Household and Business Finances | Financial Stability Review – October 2019 | RBA | 1_frs | G1 | 1 | 6 | 9 | 1_6_9 | buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. | 5 |
| 9 | 2019 | October | Household and Business Finances | Financial Stability Review – October 2019 | RBA | 1_frs | G1 | 1 | 6 | 9 | 1_6_9 | while these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. | 6 |
| 9 | 2019 | October | Household and Business Finances | Financial Stability Review – October 2019 | RBA | 1_frs | G1 | 1 | 6 | 9 | 1_6_9 | however, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. | 7 |
| 9 | 2019 | October | Household and Business Finances | Financial Stability Review – October 2019 | RBA | 1_frs | G1 | 1 | 6 | 9 | 1_6_9 | this could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. | 8 |
| 10 | 2016 | April | The Australian Financial System | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 2 | 10 | 1_2_10 | australian banks using the irb approach to credit risk have been required to disclose their leverage ratio from mid 2015. | 1 |
| 10 | 2016 | April | The Australian Financial System | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 2 | 10 | 1_2_10 | the leverage ratio is a non-risk-based measure of a bank’s tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. | 2 |
| 10 | 2016 | April | The Australian Financial System | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 2 | 10 | 1_2_10 | the leverage ratio framework is yet to be finalised internationally, although the basel committee’s governing body agreed the minimum requirement should be 3 per cent. | 3 |
| 10 | 2016 | April | The Australian Financial System | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 2 | 10 | 1_2_10 | the basel committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from january 2018. | 4 |
| 10 | 2016 | April | The Australian Financial System | Financial Stability Review – April 2016 | RBA | 1_frs | G1 | 1 | 2 | 10 | 1_2_10 | each of the australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at december 2015, well above the minimum. | 5 |
## the first sentence POS tag features
first_sentence_pos <- sentence_pos_function(first_sentence) %>% as.data.frame()
colnames(first_sentence_pos)[3:ncol(first_sentence_pos)] <- paste("sent_1st",colnames(first_sentence_pos)[3:ncol(first_sentence_pos)], sep = "_")
## the last sentence POS tag features
last_sentence_pos <- sentence_pos_function(last_sentence) %>% as.data.frame()
colnames(last_sentence_pos)[3:ncol(last_sentence_pos)] <- paste("sent_last",colnames(last_sentence_pos)[3:ncol(last_sentence_pos)], sep = "_")
sentence_pos <- left_join(first_sentence_pos, last_sentence_pos, by = "question_index")
sentence_pos %>% head() %>% kbl() %>%
kable_paper() %>%
scroll_box(width = "100%", height = "200px")| index.x | question_index | sent_1st_CC | sent_1st_DT | sent_1st_IN | sent_1st_JJ | sent_1st_JJR | sent_1st_NN | sent_1st_NNS | sent_1st_RB | sent_1st_VBN | sent_1st_VBP | sent_1st_CD | sent_1st_VBD | sent_1st_RBS | sent_1st_TO | sent_1st_VB | sent_1st_VBG | sent_1st_RP | sent_1st_VBZ | sent_1st_MD | sent_1st_PRP | sent_1st_WDT | sent_1st_RBR | sent_1st_PRP$ | sent_1st_prop_CC | sent_1st_prop_DT | sent_1st_prop_IN | sent_1st_prop_JJ | sent_1st_prop_JJR | sent_1st_prop_NN | sent_1st_prop_NNS | sent_1st_prop_RB | sent_1st_prop_VBN | sent_1st_prop_VBP | sent_1st_prop_CD | sent_1st_prop_VBD | sent_1st_prop_RBS | sent_1st_prop_TO | sent_1st_prop_VB | sent_1st_prop_VBG | sent_1st_prop_RP | sent_1st_prop_VBZ | sent_1st_prop_MD | sent_1st_prop_PRP | sent_1st_prop_WDT | sent_1st_prop_RBR | sent_1st_prop_PRP$ | index.y | sent_last_DT | sent_last_JJ | sent_last_MD | sent_last_NN | sent_last_NNS | sent_last_PRP | sent_last_RB | sent_last_TO | sent_last_VB | sent_last_VBP | sent_last_WDT | sent_last_VBN | sent_last_CC | sent_last_IN | sent_last_JJR | sent_last_RBS | sent_last_VBG | sent_last_POS | sent_last_VBZ | sent_last_VBD | sent_last_CD | sent_last_PRP$ | sent_last_RBR | sent_last_prop_DT | sent_last_prop_JJ | sent_last_prop_MD | sent_last_prop_NN | sent_last_prop_NNS | sent_last_prop_PRP | sent_last_prop_RB | sent_last_prop_TO | sent_last_prop_VB | sent_last_prop_VBP | sent_last_prop_WDT | sent_last_prop_VBN | sent_last_prop_CC | sent_last_prop_IN | sent_last_prop_JJR | sent_last_prop_RBS | sent_last_prop_VBG | sent_last_prop_POS | sent_last_prop_VBZ | sent_last_prop_VBD | sent_last_prop_CD | sent_last_prop_PRP$ | sent_last_prop_RBR |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 1_10_1 | 1 | 2 | 5 | 4 | 1 | 6 | 3 | 2 | 1 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 3.70 | 7.41 | 18.52 | 14.81 | 3.70 | 22.22 | 11.11 | 7.41 | 3.70 | 7.41 | 0.00 | 0.00 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 | 1 | 1 | 2 | 1 | 2 | 2 | 1 | 1 | 2 | 2 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 6.25 | 12.50 | 6.25 | 12.50 | 12.50 | 6.25 | 6.25 | 12.50 | 12.50 | 6.25 | 6.25 | 0.00 | 0.00 | 0.00 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0 | 0 | 0 |
| 1 | 1_4_2 | 1 | 3 | 9 | 7 | 0 | 10 | 4 | 3 | 0 | 0 | 2 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2.44 | 7.32 | 21.95 | 17.07 | 0.00 | 24.39 | 9.76 | 7.32 | 0.00 | 0.00 | 4.88 | 4.88 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 | 1 | 1 | 0 | 1 | 2 | 2 | 0 | 1 | 1 | 2 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 9.09 | 0.00 | 9.09 | 18.18 | 18.18 | 0.00 | 9.09 | 9.09 | 18.18 | 0.00 | 0.00 | 9.09 | 0.00 | 0.00 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0 | 0 | 0 |
| 1 | 1_9_3 | 1 | 1 | 3 | 4 | 1 | 1 | 4 | 2 | 1 | 1 | 0 | 0 | 1 | 1 | 2 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 4.00 | 4.00 | 12.00 | 16.00 | 4.00 | 4.00 | 16.00 | 8.00 | 4.00 | 4.00 | 0.00 | 0.00 | 4 | 4.00 | 8.00 | 8.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 | 1 | 1 | 4 | 0 | 1 | 4 | 0 | 2 | 1 | 2 | 1 | 0 | 1 | 1 | 3 | 1 | 1 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 4.00 | 16.00 | 0.00 | 4.00 | 16.00 | 0.00 | 8.00 | 4.00 | 8.00 | 4.00 | 0.00 | 4.00 | 4.00 | 12.00 | 4 | 4 | 8.00 | 0.00 | 0.00 | 0.00 | 0 | 0 | 0 |
| 1 | 1_3_4 | 2 | 3 | 4 | 3 | 0 | 6 | 2 | 0 | 1 | 0 | 0 | 0 | 0 | 2 | 1 | 2 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 7.14 | 10.71 | 14.29 | 10.71 | 0.00 | 21.43 | 7.14 | 0.00 | 3.57 | 0.00 | 0.00 | 0.00 | 0 | 7.14 | 3.57 | 7.14 | 3.57 | 3.57 | 0.00 | 0.00 | 0.00 | 0.00 | 0 | 1 | 2 | 5 | 0 | 2 | 5 | 0 | 2 | 1 | 0 | 2 | 0 | 1 | 1 | 3 | 0 | 0 | 2 | 1 | 1 | 0 | 0 | 0 | 0 | 7.14 | 17.86 | 0.00 | 7.14 | 17.86 | 0.00 | 7.14 | 3.57 | 0.00 | 7.14 | 0.00 | 3.57 | 3.57 | 10.71 | 0 | 0 | 7.14 | 3.57 | 3.57 | 0.00 | 0 | 0 | 0 |
| 1 | 1_3_5 | 1 | 0 | 5 | 4 | 0 | 5 | 5 | 2 | 0 | 2 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 3.23 | 0.00 | 16.13 | 12.90 | 0.00 | 16.13 | 16.13 | 6.45 | 0.00 | 6.45 | 0.00 | 0.00 | 0 | 3.23 | 3.23 | 3.23 | 3.23 | 0.00 | 3.23 | 3.23 | 3.23 | 0.00 | 0 | 1 | 3 | 6 | 1 | 6 | 4 | 1 | 1 | 1 | 1 | 2 | 0 | 1 | 3 | 6 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 8.11 | 16.22 | 2.70 | 16.22 | 10.81 | 2.70 | 2.70 | 2.70 | 2.70 | 5.41 | 0.00 | 2.70 | 8.11 | 16.22 | 0 | 0 | 2.70 | 0.00 | 0.00 | 0.00 | 0 | 0 | 0 |
| 1 | 1_9_6 | 1 | 3 | 4 | 2 | 1 | 5 | 0 | 1 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 4.76 | 14.29 | 19.05 | 9.52 | 4.76 | 23.81 | 0.00 | 4.76 | 9.52 | 0.00 | 0.00 | 0.00 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 4.76 | 0.00 | 0.00 | 0.00 | 4.76 | 0 | 1 | 2 | 2 | 2 | 3 | 1 | 2 | 0 | 1 | 3 | 0 | 1 | 0 | 1 | 4 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 8.70 | 8.70 | 8.70 | 13.04 | 4.35 | 8.70 | 0.00 | 4.35 | 13.04 | 0.00 | 4.35 | 0.00 | 4.35 | 17.39 | 0 | 0 | 0.00 | 0.00 | 0.00 | 4.35 | 0 | 0 | 0 |
Extract POS features for the first three words in the openning sentence of a given paragraph.
#this is the function to remove the apostrophes that are used to quote a sing word, for example 'peaking', but will keep the apostrophe that is used in the middle of a work, for example ABS's forecast, etc.
fun1 <- function(x) substr(x, 1 + (1 * as.numeric(substr(x,1,1)=="'")), nchar(x) - (1 * as.numeric(substr(x, nchar(x), nchar(x)) == "'")))
word_pos_function <- function(text_input_df, variable_choose){
text_data <- text_input_df %>% select(question_index, variable_choose)
#rename the selected word to word
colnames(text_data)[2] <- "word_choose"
text_p1 <- text_data[str_detect(text_data$word_choose, "'"),]
if(nrow(text_p1)==0){
text_p1 <- NULL
} else {
for (i in (1:nrow(text_p1))){
text_p1$word_pos[i] <- paste(
tagPOS(text_p1$word_choose[i])$POStags[1],
tagPOS(text_p1$word_choose[i])$POStags[2], sep = "_")
}
}
text_p2 <- text_data[!str_detect(text_data$word_choose, "'"),]
text_p2$word_pos <- tagPOS(text_p2$word_choose)$POStags
word_pos_result <- rbind(text_p1,text_p2)
word_pos_result <- word_pos_result %>% arrange(as.numeric(row.names(word_pos_result)))
word_pos_result
}
sentence_text <- first_sentence
first3words <- cbind(question_index = sentence_text$question_index,
start_word = word(sentence_text$sentence,1),
second_word =word(sentence_text$sentence, 2),
third_word = word(sentence_text$sentence, 3)) %>% as.data.frame()
first3words$start_word <- fun1(as.character(first3words$start_word))
first3words$second_word <- fun1(as.character(first3words$second_word))
first3words$third_word <- fun1(as.character(first3words$third_word))
# first3words$start_word <- removePunctuation(as.character(first3words$start_word))
# first3words$second_word <- removePunctuation(as.character(first3words$second_word))
# first3words$third_word <- removePunctuation(as.character(first3words$third_word))
#remove all punctuations except for the apostrophe (')
first3words$start_word <- gsub("[^[:alnum:][:space:]']", "", first3words$start_word)
first3words$second_word <- gsub("[^[:alnum:][:space:]']", "", first3words$second_word)
first3words$third_word <- gsub("[^[:alnum:][:space:]']", "", first3words$third_word)
library(NLP)
library(openNLP)
first_word_pos_df <- word_pos_function(first3words, "start_word")
second_word_pos_df <- word_pos_function(first3words, "second_word")
third_word_pos_df <- word_pos_function(first3words, "third_word")
first3words_pos_df <-
left_join (first3words, first_word_pos_df, by = "question_index") %>%
left_join(.,second_word_pos_df, by = "question_index") %>%
left_join(.,third_word_pos_df, by = "question_index")
first3words_pos_df <- rename(first3words_pos_df,
c("word_pos.x" = "word_pos.word1", "word_pos.y" = "word_pos.word2", "word_pos" = "word_pos.word3"))
#A snapshot of the output table is:
kbl(first3words_pos_df) %>%
kable_paper() %>%
scroll_box(width = "100%", height = "200px")| question_index | start_word | second_word | third_word | word_choose.x | word_pos.word1 | word_choose.y | word_pos.word2 | word_choose | word_pos.word3 |
|---|---|---|---|---|---|---|---|---|---|
| 1_10_1 | along | with | the | along | IN | with | IN | the | DT |
| 1_4_2 | total | superannuation | assets | total | JJ | superannuation | NN | assets | NNS |
| 1_9_3 | despite | challenging | economic | despite | IN | challenging | VBG | economic | JJ |
| 1_3_4 | with | the | increasing | with | IN | the | DT | increasing | VBG |
| 1_3_5 | if | financial | strains | if | IN | financial | JJ | strains | NNS |
| 1_9_6 | much | of | the | much | JJ | of | IN | the | DT |
| 1_8_7 | the | increase | in | the | DT | increase | NN | in | IN |
| 1_5_8 | housing | market | risks | housing | NN | market | NN | risks | NNS |
| 1_6_9 | personal | debt | which | personal | JJ | debt | NN | which | WDT |
| 1_2_10 | australian | banks | using | australian | NN | banks | NNS | using | VBG |
Extract the POS features for the first word of each setence in a given paragraph.
## extract the first word of first sentence
firstwords <- cbind(question_index = text_sentence$question_index, rank = text_sentence$rank,
start_word = word(text_sentence$sentence,1)) %>% as.data.frame() #4196 rows
#remove double apostrophe (such as 'yes')
firstwords$start_word <- fun1(as.character(firstwords$start_word))
#remove all punctuations except for the apostrophe (')
firstwords$start_word <- gsub("[^[:alnum:][:space:]']", "", firstwords$start_word)
text_input_df <- firstwords
variable_choose <- "start_word"
text_data <- text_input_df %>% select(question_index, rank, variable_choose) %>% filter(nchar(start_word) > 0)
#rename the selected word to word_choose
colnames(text_data)[3] <- "word_choose"
text_p1 <- text_data[str_detect(text_data$word_choose, "'"),] #find the sentences beginning with "'"
if(nrow(text_p1)==0){
text_p1 <- NULL
} else {
for (i in (1:nrow(text_p1))){
text_p1$word_pos[i] <- paste(
tagPOS(text_p1$word_choose[i])$POStags[1],
tagPOS(text_p1$word_choose[i])$POStags[2], sep = "_")
}
}
text_p2 <- text_data[!str_detect(text_data$word_choose, "'"),]
text_p2$word_pos <- tagPOS(text_p2$word_choose)$POStags
word_pos_result <- rbind(text_p1,text_p2)
word_pos_result <- word_pos_result %>% arrange(as.numeric(row.names(word_pos_result)))
firstwordsall <- word_pos_result
## count the PoS tag for the first word of each sentence
allsent_firstwords <- firstwordsall %>% select(-word_choose, -rank) %>% group_by(question_index) %>% count(word_pos) %>% spread(word_pos, n)
colnames(allsent_firstwords)[2:ncol(allsent_firstwords)] <- paste("sent_1st_word",colnames(allsent_firstwords)[2:ncol(allsent_firstwords)], sep = "_")
#replacing NAs with 0
allsent_firstwords[is.na(allsent_firstwords)] <- 0
#A snapshot of the output table is:
kbl(allsent_firstwords) %>%
kable_paper() %>%
scroll_box(width = "100%", height = "200px")| question_index | sent_1st_word_CC | sent_1st_word_DT | sent_1st_word_IN | sent_1st_word_JJ | sent_1st_word_JJR | sent_1st_word_JJS | sent_1st_word_NN | sent_1st_word_PRP | sent_1st_word_RB |
|---|---|---|---|---|---|---|---|---|---|
| 1_10_1 | 0 | 1 | 2 | 0 | 0 | 0 | 0 | 1 | 0 |
| 1_2_10 | 0 | 4 | 0 | 1 | 0 | 0 | 0 | 0 | 0 |
| 1_3_4 | 0 | 0 | 2 | 0 | 0 | 0 | 1 | 0 | 0 |
| 1_3_5 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 |
| 1_4_2 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 0 |
| 1_5_8 | 1 | 1 | 1 | 0 | 0 | 0 | 4 | 0 | 0 |
| 1_6_9 | 0 | 1 | 3 | 1 | 1 | 0 | 1 | 0 | 1 |
| 1_8_7 | 0 | 4 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| 1_9_3 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| 1_9_6 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 0 |
Using a list of clue words listed in Robin Cohen (1984) to generate argumentative features including:
A snapshot of the output table is shown as below.
clue_words <- read.csv("./data_input/clue_words.csv")
sentence_clue_feature <- clue_words_feature_function(text_sentence) %>% as.data.frame()
text_feature_pos_clue <-
left_join(survey_feature_part2, first_word_pos_df, by = "question_index") %>%
left_join(., first3words_pos_df, by = "question_index") %>%
left_join(.,sentence_clue_feature, by = "question_index") %>%
left_join(.,allsent_firstwords, by = "question_index") %>%
select(-start_word,-second_word, -third_word) #CC.X is for the whole paragraph; CC.y is for the sentence 1 features of POS (count).
#A snapshot of the output table is:
kbl(text_feature_pos_clue) %>%
kable_paper() %>%
scroll_box(width = "100%", height = "200px")| X | year | month | issue | paragraph | source | source_group | survey_group | question_group | index.x | question_index | paragraph_clean | word_count_stats | sentence_count | readability_stats.sylls | readability_stats.polys | fk_grade_level | FRES_score | comma_count | punc_count | digit_count | index.y | CC | DT | IN | JJ | JJR | MD | NN | NNS | PRP | PRP$ | RB | RBR | TO | VB | VBD | VBG | VBN | VBP | VBZ | WDT | CD | RBS | POS | RP | JJS | NNP | EX | pos_prop_CC | pos_prop_DT | pos_prop_IN | pos_prop_JJ | pos_prop_JJR | pos_prop_MD | pos_prop_NN | pos_prop_NNS | pos_prop_PRP | pos_prop_PRP$ | pos_prop_RB | pos_prop_RBR | pos_prop_TO | pos_prop_VB | pos_prop_VBD | pos_prop_VBG | pos_prop_VBN | pos_prop_VBP | pos_prop_VBZ | pos_prop_WDT | pos_prop_CD | pos_prop_RBS | pos_prop_POS | pos_prop_RP | pos_prop_JJS | pos_prop_NNP | pos_prop_EX | word_choose.x.x | word_pos | word_choose.x | word_pos.word1 | word_choose.y | word_pos.word2 | word_choose.y.y | word_pos.word3 | sent1st_clue_Attitudinal | sent1st_clue_connective | sentlast_clue_Attitudinal | sentlast_clue_connective | sentlast_clue_Contrast | sentlast_clue_summary | sentmiddle_clue_Attitudinal | sentmiddle_clue_connective | sentmiddle_clue_detail | sentmiddle_clue_inference | sent_1st_word_CC | sent_1st_word_DT | sent_1st_word_IN | sent_1st_word_JJ | sent_1st_word_JJR | sent_1st_word_JJS | sent_1st_word_NN | sent_1st_word_PRP | sent_1st_word_RB |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. | 1_frs | G1 | 1 | 10 | 1 | 1_10_1 | Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. | 92 | 4 | 163 | 17 | 14.28652 | 33.60087 | 3 | 9 | 0 | 1 | 5 | 6 | 8 | 13 | 1 | 3 | 14 | 12 | 2 | 1 | 5 | 1 | 4 | 6 | 2 | 1 | 2 | 3 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 5.49 | 6.59 | 8.79 | 14.29 | 1.10 | 3.30 | 15.38 | 13.19 | 2.20 | 1.10 | 5.49 | 1.10 | 4.40 | 6.59 | 2.20 | 1.10 | 2.20 | 3.30 | 1.10 | 1.10 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | along | IN | along | IN | with | IN | the | DT | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 2 | 0 | 0 | 0 | 0 | 1 | 0 |
| 2 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. | 1_frs | G1 | 1 | 4 | 2 | 1_4_2 | Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. | 82 | 3 | 150 | 15 | 16.65537 | 24.33557 | 3 | 3 | 5 | 2 | 2 | 6 | 13 | 11 | 1 | 2 | 16 | 12 | 1 | 0 | 5 | 1 | 2 | 3 | 2 | 0 | 2 | 2 | 1 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 2.38 | 7.14 | 15.48 | 13.10 | 1.19 | 2.38 | 19.05 | 14.29 | 1.19 | 0.00 | 5.95 | 1.19 | 2.38 | 3.57 | 2.38 | 0.00 | 2.38 | 2.38 | 1.19 | 0.00 | 2.38 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | total | JJ | total | JJ | superannuation | NN | assets | NNS | 2 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 0 |
| 3 | 2017 | October | The Global Financial Environment | Financial Stability Review – October 2017 | RBA | Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. | 1_frs | G1 | 1 | 9 | 3 | 1_9_3 | Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. | 25 | 1 | 54 | 8 | 19.64800 | -1.27600 | 1 | 1 | 0 | 3 | 1 | 1 | 3 | 4 | 1 | 0 | 1 | 4 | 0 | 0 | 2 | 0 | 1 | 2 | 0 | 2 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 4.00 | 4.00 | 12.00 | 16.00 | 4.00 | 0.00 | 4.00 | 16.00 | 0.00 | 0.00 | 8.00 | 0.00 | 4.00 | 8.00 | 0.00 | 8.00 | 4.00 | 4.00 | 0.00 | 0.00 | 0.00 | 4.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | despite | IN | despite | IN | challenging | VBG | economic | JJ | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| 4 | 2016 | October | The Global Financial Environment | Financial Stability Review – October 2016 | RBA | With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. | 1_frs | G1 | 1 | 3 | 4 | 1_3_4 | With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. | 73 | 3 | 143 | 20 | 17.01507 | 16.41338 | 5 | 4 | 0 | 4 | 4 | 5 | 10 | 12 | 0 | 0 | 14 | 11 | 0 | 0 | 2 | 0 | 3 | 1 | 0 | 3 | 2 | 3 | 2 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 5.41 | 6.76 | 13.51 | 16.22 | 0.00 | 0.00 | 18.92 | 14.86 | 0.00 | 0.00 | 2.70 | 0.00 | 4.05 | 1.35 | 0.00 | 4.05 | 2.70 | 4.05 | 2.70 | 0.00 | 0.00 | 0.00 | 1.35 | 1.35 | 0.00 | 0.00 | 0.00 | with | IN | with | IN | the | DT | increasing | VBG | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | 0 | 0 | 2 | 0 | 0 | 0 | 1 | 0 | 0 |
| 5 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. | 1_frs | G1 | 1 | 3 | 5 | 1_3_5 | If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. | 89 | 3 | 163 | 21 | 17.59124 | 21.78176 | 3 | 6 | 0 | 5 | 5 | 3 | 15 | 16 | 0 | 2 | 15 | 11 | 2 | 0 | 5 | 0 | 2 | 2 | 0 | 2 | 1 | 5 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 5.62 | 3.37 | 16.85 | 17.98 | 0.00 | 2.25 | 16.85 | 12.36 | 2.25 | 0.00 | 5.62 | 0.00 | 2.25 | 2.25 | 0.00 | 2.25 | 1.12 | 5.62 | 1.12 | 0.00 | 0.00 | 0.00 | 1.12 | 1.12 | 0.00 | 0.00 | 0.00 | if | IN | if | IN | financial | JJ | strains | NNS | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 |
| 6 | 2018 | October | The Global Financial Environment | Financial Stability Review – October 2018 | RBA | Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. | 1_frs | G1 | 1 | 9 | 6 | 1_9_6 | Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. | 118 | 6 | 200 | 19 | 12.08000 | 43.48350 | 6 | 10 | 0 | 6 | 8 | 14 | 14 | 9 | 1 | 2 | 28 | 10 | 3 | 0 | 2 | 1 | 6 | 4 | 1 | 1 | 6 | 2 | 4 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 6.78 | 11.86 | 11.86 | 7.63 | 0.85 | 1.69 | 23.73 | 8.47 | 2.54 | 0.00 | 1.69 | 0.85 | 5.08 | 3.39 | 0.85 | 0.85 | 5.08 | 1.69 | 3.39 | 0.85 | 0.00 | 0.85 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | much | JJ | much | JJ | of | IN | the | DT | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 0 |
| 7 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. | 1_frs | G1 | 1 | 8 | 7 | 1_8_7 | The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. | 126 | 5 | 237 | 31 | 16.43324 | 22.12843 | 5 | 11 | 11 | 7 | 2 | 18 | 18 | 13 | 1 | 2 | 33 | 9 | 0 | 2 | 5 | 0 | 4 | 4 | 3 | 1 | 6 | 0 | 5 | 0 | 5 | 0 | 4 | 0 | 0 | 0 | 0 | 1.48 | 13.33 | 13.33 | 9.63 | 0.74 | 1.48 | 24.44 | 6.67 | 0.00 | 1.48 | 3.70 | 0.00 | 2.96 | 2.96 | 2.22 | 0.74 | 4.44 | 0.00 | 3.70 | 0.00 | 3.70 | 0.00 | 2.96 | 0.00 | 0.00 | 0.00 | 0.00 | the | DT | the | DT | increase | NN | in | IN | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 4 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| 8 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. | 1_frs | G1 | 1 | 5 | 8 | 1_5_8 | Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. | 176 | 8 | 324 | 33 | 14.71273 | 28.76409 | 11 | 15 | 16 | 8 | 8 | 11 | 27 | 24 | 2 | 1 | 36 | 21 | 0 | 4 | 9 | 1 | 4 | 2 | 1 | 4 | 9 | 4 | 2 | 1 | 4 | 0 | 0 | 0 | 2 | 1 | 0 | 4.49 | 6.18 | 15.17 | 13.48 | 1.12 | 0.56 | 20.22 | 11.80 | 0.00 | 2.25 | 5.06 | 0.56 | 2.25 | 1.12 | 0.56 | 2.25 | 5.06 | 2.25 | 1.12 | 0.56 | 2.25 | 0.00 | 0.00 | 0.00 | 1.12 | 0.56 | 0.00 | housing | NN | housing | NN | market | NN | risks | NNS | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 4 | 0 | 0 |
| 9 | 2019 | October | Household and Business Finances | Financial Stability Review – October 2019 | RBA | Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. | 1_frs | G1 | 1 | 6 | 9 | 1_6_9 | Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. | 211 | 8 | 387 | 53 | 16.33890 | 24.89755 | 12 | 17 | 0 | 9 | 9 | 15 | 28 | 28 | 2 | 3 | 39 | 32 | 1 | 0 | 7 | 2 | 7 | 7 | 0 | 10 | 5 | 11 | 2 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 4.27 | 7.11 | 13.27 | 13.27 | 0.95 | 1.42 | 18.48 | 15.17 | 0.47 | 0.00 | 3.32 | 0.95 | 3.32 | 3.32 | 0.00 | 4.74 | 2.37 | 5.21 | 0.95 | 0.95 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.47 | personal | JJ | personal | JJ | debt | NN | which | WDT | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | 0 | 1 | 3 | 1 | 1 | 0 | 1 | 0 | 1 |
| 10 | 2016 | April | The Australian Financial System | Financial Stability Review – April 2016 | RBA | Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. | 1_frs | G1 | 1 | 2 | 10 | 1_2_10 | Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. | 121 | 5 | 228 | 31 | 16.08271 | 22.86043 | 4 | 10 | 19 | 10 | 1 | 19 | 12 | 9 | 0 | 1 | 35 | 5 | 0 | 2 | 4 | 0 | 11 | 6 | 3 | 3 | 5 | 1 | 4 | 0 | 7 | 0 | 2 | 0 | 0 | 0 | 0 | 0.77 | 14.62 | 9.23 | 6.92 | 0.00 | 0.77 | 26.92 | 3.85 | 0.00 | 1.54 | 3.08 | 0.00 | 8.46 | 4.62 | 2.31 | 2.31 | 3.85 | 0.77 | 3.08 | 0.00 | 5.38 | 0.00 | 1.54 | 0.00 | 0.00 | 0.00 | 0.00 | australian | NN | australian | NN | banks | NNS | using | VBG | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | 0 | 4 | 0 | 1 | 0 | 0 | 0 | 0 | 0 |
In this section, we decompose each sentence into a parse tree following the Peenn Treebank syntactic tagset introduced by Taylor, Marcus and Santorini (2003). After that, a number of syntactic structured features embedded in a parse tree are generated, such as:
Decompose each sentence of a given paragraph into a parse tree and then summarise the parse tree features for each paragraph using the openNLP package.A snapshot of the output table is:
# input the pre-defined function. Please be aware that this program requires a large memeory to run, if it doesn't run successfully, please go to the code itself and run it seperately
source("./r_function/tree_parse_feature_extract.R")
##combine the two prase data together
parse_sum <- parse_feature
#A snapshot of the output table is:
parse_sum %>% head() %>%
kbl() %>%
kable_paper() %>%
scroll_box(width = "100%", height = "200px")| ADJP | ADVP | NP | PP | S | SBAR | VP | WHNP | question_index | rank |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 1 | 8 | 5 | 2 | 0 | 3 | 0 | 1_10_1 | 1 |
| 1 | 0 | 3 | 0 | 2 | 0 | 4 | 0 | 1_10_1 | 2 |
| 1 | 0 | 10 | 2 | 6 | 1 | 10 | 0 | 1_10_1 | 3 |
| 0 | 1 | 6 | 1 | 3 | 1 | 5 | 1 | 1_10_1 | 4 |
| 1 | 1 | 15 | 8 | 2 | 1 | 2 | 0 | 1_4_2 | 1 |
| 3 | 0 | 12 | 3 | 4 | 2 | 6 | 0 | 1_4_2 | 2 |
After extracting the parse tree related features for each sentence of sample paragraphs. We constructed three types of parse-related features for each paragraph:
A snapshot of the output table is shown as below:
#create a rank index for each paragraph
parse_sum <- parse_sum %>% group_by (question_index) %>% mutate(rank = c(1:n()))
##create the parse feature for sentence 1
parse_1stsent_feature <- parse_sum %>% dplyr::filter(rank==1)%>% dplyr::select(-rank)
##rename the columns
colnames(parse_1stsent_feature)[1:ncol(parse_1stsent_feature)-1] <-
paste("sent_1st_parse",colnames(parse_1stsent_feature)[1:ncol(parse_1stsent_feature)-1], sep = "_")
library(plyr)
##create the parse feature for the last sentence
parse_lastsent_feature <- parse_sum%>% dplyr::group_by(question_index) %>% dplyr::filter(rank==max(rank))%>%
dplyr::select(-rank)
colnames(parse_lastsent_feature)[2:ncol(parse_lastsent_feature)] <- paste("sent_last_parse",colnames(parse_lastsent_feature)[2:ncol(parse_lastsent_feature)], sep = "_")
colnames(parse_lastsent_feature)[1:ncol(parse_lastsent_feature)-1] <-
paste("sent_last_parse",colnames(parse_lastsent_feature)[1:ncol(parse_lastsent_feature)-1], sep = "_")
parse_lastsent_feature <- plyr::rename(parse_lastsent_feature, c("sent_last_parse_question_index" = "question_index"))
##create the parse feature for a paragraph (all sentences included)
parse_para_feature <- parse_sum %>% dplyr::select(-rank) %>% dplyr::group_by(question_index) %>%
dplyr::summarise_each(dplyr::funs(sum))
#create the final table for the parse feature
parse_feature <- dplyr::left_join(parse_1stsent_feature, parse_lastsent_feature, by="question_index") %>%
dplyr::left_join(.,parse_para_feature,by="question_index")
#.x is the features for the first sentence, y.is the parse features for the last sentence, and the final section is the features for the whole paragraph
#replace NAs with 0
parse_feature[is.na(parse_feature)] <- 0
#A snapshot of the output table is:
parse_feature %>% head() %>%
kbl() %>%
kable_paper() %>%
scroll_box(width = "100%", height = "200px")| sent_1st_parse_ADJP | sent_1st_parse_ADVP | sent_1st_parse_NP | sent_1st_parse_PP | sent_1st_parse_S | sent_1st_parse_SBAR | sent_1st_parse_VP | sent_1st_parse_WHNP | question_index | sent_last_parse_ADJP | sent_last_parse_sent_last_parse_ADVP | sent_last_parse_sent_last_parse_NP | sent_last_parse_sent_last_parse_PP | sent_last_parse_sent_last_parse_S | sent_last_parse_sent_last_parse_SBAR | sent_last_parse_sent_last_parse_VP | sent_last_parse_sent_last_parse_WHNP | ADJP | ADVP | NP | PP | S | SBAR | VP | WHNP |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 1 | 8 | 5 | 2 | 0 | 3 | 0 | 1_10_1 | 0 | 1 | 6 | 1 | 3 | 1 | 5 | 1 | 3 | 2 | 27 | 8 | 13 | 2 | 22 | 1 |
| 1 | 1 | 15 | 8 | 2 | 1 | 2 | 0 | 1_4_2 | 0 | 0 | 2 | 0 | 2 | 0 | 4 | 0 | 4 | 1 | 29 | 11 | 8 | 3 | 12 | 0 |
| 2 | 1 | 6 | 3 | 2 | 0 | 5 | 0 | 1_9_3 | 2 | 1 | 6 | 3 | 2 | 0 | 5 | 0 | 2 | 1 | 6 | 3 | 2 | 0 | 5 | 0 |
| 0 | 0 | 13 | 5 | 2 | 1 | 4 | 0 | 1_3_4 | 2 | 1 | 6 | 3 | 5 | 1 | 5 | 0 | 2 | 1 | 27 | 11 | 8 | 2 | 10 | 0 |
| 0 | 0 | 14 | 5 | 2 | 1 | 3 | 0 | 1_3_5 | 1 | 1 | 15 | 5 | 3 | 1 | 7 | 0 | 3 | 2 | 37 | 12 | 7 | 3 | 12 | 0 |
| 3 | 0 | 7 | 4 | 1 | 0 | 3 | 0 | 1_9_6 | 0 | 0 | 7 | 3 | 4 | 2 | 7 | 1 | 5 | 1 | 44 | 16 | 11 | 3 | 24 | 1 |
Now, we can create the final text feature table that including all textual features, POS tag feature and syntactic structured features over parse trees. The view of the final output table is shown as below.
text_feature_base <- left_join(text_feature_pos_clue, parse_feature, by="question_index")
#take a look of data
# saveRDS(text_feature_base, "smp_2021_text_feature.rds")
#replace NAs with 0
text_feature_base[is.na(text_feature_base)] <- 0
kbl(text_feature_base) %>%
kable_paper() %>%
scroll_box(width = "100%", height = "200px")| X | year | month | issue | paragraph | source | source_group | survey_group | question_group | index.x | question_index | paragraph_clean | word_count_stats | sentence_count | readability_stats.sylls | readability_stats.polys | fk_grade_level | FRES_score | comma_count | punc_count | digit_count | index.y | CC | DT | IN | JJ | JJR | MD | NN | NNS | PRP | PRP$ | RB | RBR | TO | VB | VBD | VBG | VBN | VBP | VBZ | WDT | CD | RBS | POS | RP | JJS | NNP | EX | pos_prop_CC | pos_prop_DT | pos_prop_IN | pos_prop_JJ | pos_prop_JJR | pos_prop_MD | pos_prop_NN | pos_prop_NNS | pos_prop_PRP | pos_prop_PRP$ | pos_prop_RB | pos_prop_RBR | pos_prop_TO | pos_prop_VB | pos_prop_VBD | pos_prop_VBG | pos_prop_VBN | pos_prop_VBP | pos_prop_VBZ | pos_prop_WDT | pos_prop_CD | pos_prop_RBS | pos_prop_POS | pos_prop_RP | pos_prop_JJS | pos_prop_NNP | pos_prop_EX | word_choose.x.x | word_pos | word_choose.x | word_pos.word1 | word_choose.y | word_pos.word2 | word_choose.y.y | word_pos.word3 | sent1st_clue_Attitudinal | sent1st_clue_connective | sentlast_clue_Attitudinal | sentlast_clue_connective | sentlast_clue_Contrast | sentlast_clue_summary | sentmiddle_clue_Attitudinal | sentmiddle_clue_connective | sentmiddle_clue_detail | sentmiddle_clue_inference | sent_1st_word_CC | sent_1st_word_DT | sent_1st_word_IN | sent_1st_word_JJ | sent_1st_word_JJR | sent_1st_word_JJS | sent_1st_word_NN | sent_1st_word_PRP | sent_1st_word_RB | sent_1st_parse_ADJP | sent_1st_parse_ADVP | sent_1st_parse_NP | sent_1st_parse_PP | sent_1st_parse_S | sent_1st_parse_SBAR | sent_1st_parse_VP | sent_1st_parse_WHNP | sent_last_parse_ADJP | sent_last_parse_sent_last_parse_ADVP | sent_last_parse_sent_last_parse_NP | sent_last_parse_sent_last_parse_PP | sent_last_parse_sent_last_parse_S | sent_last_parse_sent_last_parse_SBAR | sent_last_parse_sent_last_parse_VP | sent_last_parse_sent_last_parse_WHNP | ADJP | ADVP | NP | PP | S | SBAR | VP | WHNP |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. | 1_frs | G1 | 1 | 10 | 1 | 1_10_1 | Along with the increase in shadow lending, banks – especially small and medium-sized banks – have also sourced more funding from the short-term interbank market over recent years. This has increased their liquidity risks and made them even more interconnected and systemic. If corporate defaults were to rise, investors and creditor banks may be reluctant to roll over such short-term funding, and so the interbank market could exacerbate financial problems at the banks bearing loan losses. It could also transmit distress to other institutions that investors consider to have a similar vulnerability. | 92 | 4 | 163 | 17 | 14.28652 | 33.60087 | 3 | 9 | 0 | 1 | 5 | 6 | 8 | 13 | 1 | 3 | 14 | 12 | 2 | 1 | 5 | 1 | 4 | 6 | 2 | 1 | 2 | 3 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 5.49 | 6.59 | 8.79 | 14.29 | 1.10 | 3.30 | 15.38 | 13.19 | 2.20 | 1.10 | 5.49 | 1.10 | 4.40 | 6.59 | 2.20 | 1.10 | 2.20 | 3.30 | 1.10 | 1.10 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | along | IN | along | IN | with | IN | the | DT | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 2 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 8 | 5 | 2 | 0 | 3 | 0 | 0 | 1 | 6 | 1 | 3 | 1 | 5 | 1 | 3 | 2 | 27 | 8 | 13 | 2 | 22 | 1 |
| 2 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. | 1_frs | G1 | 1 | 4 | 2 | 1_4_2 | Total superannuation assets grew at an annualised rate of nearly 5 per cent over the first half of 2016, somewhat below the average pace of recent years, as low bond yields and relatively subdued equity market returns weighed on investment income. While net contributions have remained fairly stable in recent years, it is likely that outflows will trend higher relative to contributions as the population ages and more members enter the drawdown phase. Superannuation funds will therefore need to consider the associated liquidity implications. | 82 | 3 | 150 | 15 | 16.65537 | 24.33557 | 3 | 3 | 5 | 2 | 2 | 6 | 13 | 11 | 1 | 2 | 16 | 12 | 1 | 0 | 5 | 1 | 2 | 3 | 2 | 0 | 2 | 2 | 1 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 2.38 | 7.14 | 15.48 | 13.10 | 1.19 | 2.38 | 19.05 | 14.29 | 1.19 | 0.00 | 5.95 | 1.19 | 2.38 | 3.57 | 2.38 | 0.00 | 2.38 | 2.38 | 1.19 | 0.00 | 2.38 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | total | JJ | total | JJ | superannuation | NN | assets | NNS | 2 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 15 | 8 | 2 | 1 | 2 | 0 | 0 | 0 | 2 | 0 | 2 | 0 | 4 | 0 | 4 | 1 | 29 | 11 | 8 | 3 | 12 | 0 |
| 3 | 2017 | October | The Global Financial Environment | Financial Stability Review – October 2017 | RBA | Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. | 1_frs | G1 | 1 | 9 | 3 | 1_9_3 | Despite challenging economic conditions in recent years, banking systems in the larger emerging market economies are generally profitable and most appear to be well capitalised. | 25 | 1 | 54 | 8 | 19.64800 | -1.27600 | 1 | 1 | 0 | 3 | 1 | 1 | 3 | 4 | 1 | 0 | 1 | 4 | 0 | 0 | 2 | 0 | 1 | 2 | 0 | 2 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 4.00 | 4.00 | 12.00 | 16.00 | 4.00 | 0.00 | 4.00 | 16.00 | 0.00 | 0.00 | 8.00 | 0.00 | 4.00 | 8.00 | 0.00 | 8.00 | 4.00 | 4.00 | 0.00 | 0.00 | 0.00 | 4.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | despite | IN | despite | IN | challenging | VBG | economic | JJ | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 1 | 6 | 3 | 2 | 0 | 5 | 0 | 2 | 1 | 6 | 3 | 2 | 0 | 5 | 0 | 2 | 1 | 6 | 3 | 2 | 0 | 5 | 0 |
| 4 | 2016 | October | The Global Financial Environment | Financial Stability Review – October 2016 | RBA | With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. | 1_frs | G1 | 1 | 3 | 4 | 1_3_4 | With the increasing size and integration of emerging markets in the global economy and financial system, the potential for distress to spill over to other economies has risen. As for China, transmission channels include direct financial links, trade links and risk sentiment in international financial markets. Lending to emerging markets by advanced economy banks has increased significantly over the past decade and, while overall exposures are relatively small, some banks’ exposures are significant. | 73 | 3 | 143 | 20 | 17.01507 | 16.41338 | 5 | 4 | 0 | 4 | 4 | 5 | 10 | 12 | 0 | 0 | 14 | 11 | 0 | 0 | 2 | 0 | 3 | 1 | 0 | 3 | 2 | 3 | 2 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 5.41 | 6.76 | 13.51 | 16.22 | 0.00 | 0.00 | 18.92 | 14.86 | 0.00 | 0.00 | 2.70 | 0.00 | 4.05 | 1.35 | 0.00 | 4.05 | 2.70 | 4.05 | 2.70 | 0.00 | 0.00 | 0.00 | 1.35 | 1.35 | 0.00 | 0.00 | 0.00 | with | IN | with | IN | the | DT | increasing | VBG | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 13 | 5 | 2 | 1 | 4 | 0 | 2 | 1 | 6 | 3 | 5 | 1 | 5 | 0 | 2 | 1 | 27 | 11 | 8 | 2 | 10 | 0 |
| 5 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. | 1_frs | G1 | 1 | 3 | 5 | 1_3_5 | If financial strains that threaten growth in China emerge, they could spill over to other economies by affecting trade volumes and commodity prices, as well as sentiment in global financial markets. Direct financial linkages between China and other economies are small in aggregate because China’s capital account is still relatively closed. But these linkages have grown – both in terms of foreign bank lending to China and Chinese bank lending abroad – and are sizeable for particular jurisdictions, so they could be an additional mechanism for transmitting financial difficulties. | 89 | 3 | 163 | 21 | 17.59124 | 21.78176 | 3 | 6 | 0 | 5 | 5 | 3 | 15 | 16 | 0 | 2 | 15 | 11 | 2 | 0 | 5 | 0 | 2 | 2 | 0 | 2 | 1 | 5 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 5.62 | 3.37 | 16.85 | 17.98 | 0.00 | 2.25 | 16.85 | 12.36 | 2.25 | 0.00 | 5.62 | 0.00 | 2.25 | 2.25 | 0.00 | 2.25 | 1.12 | 5.62 | 1.12 | 0.00 | 0.00 | 0.00 | 1.12 | 1.12 | 0.00 | 0.00 | 0.00 | if | IN | if | IN | financial | JJ | strains | NNS | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 14 | 5 | 2 | 1 | 3 | 0 | 1 | 1 | 15 | 5 | 3 | 1 | 7 | 0 | 3 | 2 | 37 | 12 | 7 | 3 | 12 | 0 |
| 6 | 2018 | October | The Global Financial Environment | Financial Stability Review – October 2018 | RBA | Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. | 1_frs | G1 | 1 | 9 | 6 | 1_9_6 | Much of the run-up in debt in the post-crisis period has been facilitated by the less regulated and less transparent NBFIs. Most of this lending is ultimately funded by the banking sector. While this lending has some benefits, it has allowed banks to circumvent restrictions on lending to riskier sectors and to arbitrage regulatory capital requirements. The riskier nature of the lending, and the obscure and complex interconnections between NBFIs and the banking sector, have led to the build-up of considerable credit, liquidity and contagion risks. Loan losses and defaults have been modest to date. But if they were to escalate, it could result in funding pressures in the non-bank sector, which could cascade through the financial system. | 118 | 6 | 200 | 19 | 12.08000 | 43.48350 | 6 | 10 | 0 | 6 | 8 | 14 | 14 | 9 | 1 | 2 | 28 | 10 | 3 | 0 | 2 | 1 | 6 | 4 | 1 | 1 | 6 | 2 | 4 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 6.78 | 11.86 | 11.86 | 7.63 | 0.85 | 1.69 | 23.73 | 8.47 | 2.54 | 0.00 | 1.69 | 0.85 | 5.08 | 3.39 | 0.85 | 0.85 | 5.08 | 1.69 | 3.39 | 0.85 | 0.00 | 0.85 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | much | JJ | much | JJ | of | IN | the | DT | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 3 | 0 | 7 | 4 | 1 | 0 | 3 | 0 | 0 | 0 | 7 | 3 | 4 | 2 | 7 | 1 | 5 | 1 | 44 | 16 | 11 | 3 | 24 | 1 |
| 7 | 2016 | October | The Australian Financial System | Financial Stability Review – October 2016 | RBA | The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. | 1_frs | G1 | 1 | 8 | 7 | 1_8_7 | The increase in capital ratios over the past year has also been reflected in higher leverage ratios, given that the average risk weight of their assets was largely unchanged. The leverage ratio is a non-risk based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent and that the leverage ratio should be effective from January 2018. Each of the major banks’ leverage ratios was around 5 per cent at June 2016, well above that minimum. At this level, the major Australian banks’ leverage ratio sits around the median of international banks. | 126 | 5 | 237 | 31 | 16.43324 | 22.12843 | 5 | 11 | 11 | 7 | 2 | 18 | 18 | 13 | 1 | 2 | 33 | 9 | 0 | 2 | 5 | 0 | 4 | 4 | 3 | 1 | 6 | 0 | 5 | 0 | 5 | 0 | 4 | 0 | 0 | 0 | 0 | 1.48 | 13.33 | 13.33 | 9.63 | 0.74 | 1.48 | 24.44 | 6.67 | 0.00 | 1.48 | 3.70 | 0.00 | 2.96 | 2.96 | 2.22 | 0.74 | 4.44 | 0.00 | 3.70 | 0.00 | 3.70 | 0.00 | 2.96 | 0.00 | 0.00 | 0.00 | 0.00 | the | DT | the | DT | increase | NN | in | IN | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 4 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 8 | 5 | 2 | 1 | 4 | 0 | 0 | 0 | 6 | 3 | 1 | 0 | 1 | 0 | 3 | 4 | 38 | 16 | 11 | 5 | 21 | 0 |
| 8 | 2016 | April | The Global Financial Environment | Financial Stability Review – April 2016 | RBA | Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. | 1_frs | G1 | 1 | 5 | 8 | 1_5_8 | Housing market risks are also present in some emerging market and Asian economies. This reflects large increases in residential property prices over recent years – including in Hong Kong, Brazil, Malaysia, Taiwan and Turkey – alongside increased household indebtedness. Price growth has moderated more recently and prices have fallen in some economies, including Brazil, Russia and Taiwan, which could add to the challenges already faced by these economies and their banks from weaker corporate sectors. Housing prices in Hong Kong rose especially quickly until late 2015, partly as a result of low interest rates associated with its fixed exchange rate system. But prices have fallen recently amid concerns about economic conditions in China and slower credit growth. Housing transaction volumes have also fallen, to be at their lowest level since at least the mid 1990s. Despite the slowdown in the housing market, the Hong Kong Monetary Authority imposed a countercyclical capital buffer of 0.625 per cent in January 2016, with further increases scheduled, largely in response to elevated ratios of credit-to-GDP and housing prices-to-rents relative to their long-run trends. | 176 | 8 | 324 | 33 | 14.71273 | 28.76409 | 11 | 15 | 16 | 8 | 8 | 11 | 27 | 24 | 2 | 1 | 36 | 21 | 0 | 4 | 9 | 1 | 4 | 2 | 1 | 4 | 9 | 4 | 2 | 1 | 4 | 0 | 0 | 0 | 2 | 1 | 0 | 4.49 | 6.18 | 15.17 | 13.48 | 1.12 | 0.56 | 20.22 | 11.80 | 0.00 | 2.25 | 5.06 | 0.56 | 2.25 | 1.12 | 0.56 | 2.25 | 5.06 | 2.25 | 1.12 | 0.56 | 2.25 | 0.00 | 0.00 | 0.00 | 1.12 | 0.56 | 0.00 | housing | NN | housing | NN | market | NN | risks | NNS | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 4 | 0 | 0 | 1 | 1 | 4 | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 19 | 9 | 1 | 0 | 2 | 0 | 2 | 9 | 64 | 29 | 11 | 1 | 20 | 1 |
| 9 | 2019 | October | Household and Business Finances | Financial Stability Review – October 2019 | RBA | Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. | 1_frs | G1 | 1 | 6 | 9 | 1_6_9 | Personal debt, which includes personal loans, credit card debt and other revolving credit such as margin loans, accounts for a small and declining share of household credit. In recent decades, homeowners have increasingly been able to use housing-secured financing in place of personal debt. In part, this reflects the increased availability and use of redraw facilities and offset accounts linked to residential mortgage loans. More recently, the increased use of buy-now-pay-later services may be contributing to a decline in credit card balances accruing interest. Buy-now-pay-later products are attractive to consumers because they offer the ability to smooth consumption at limited or no cost: these obligations do not incur interest, although late fees are charged if payments are missed and some providers charge regular account keeping or payment processing fees. While these products are not subject to responsible lending laws, the providers do employ some varying methods of managing risk, for example, by setting low purchase limits for new customers or requiring full repayments of previous purchases before funding new purchases. However, there are currently few safeguards that would prevent vulnerable consumers from entering into multiple arrangements with different providers. This could contribute to an increase in financial stress for some households, with lower income and/or younger households potentially more at risk. | 211 | 8 | 387 | 53 | 16.33890 | 24.89755 | 12 | 17 | 0 | 9 | 9 | 15 | 28 | 28 | 2 | 3 | 39 | 32 | 1 | 0 | 7 | 2 | 7 | 7 | 0 | 10 | 5 | 11 | 2 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 4.27 | 7.11 | 13.27 | 13.27 | 0.95 | 1.42 | 18.48 | 15.17 | 0.47 | 0.00 | 3.32 | 0.95 | 3.32 | 3.32 | 0.00 | 4.74 | 2.37 | 5.21 | 0.95 | 0.95 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.47 | personal | JJ | personal | JJ | debt | NN | which | WDT | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 3 | 1 | 1 | 0 | 1 | 0 | 1 | 0 | 0 | 14 | 3 | 1 | 1 | 1 | 1 | 0 | 1 | 11 | 5 | 1 | 0 | 2 | 0 | 3 | 4 | 82 | 29 | 24 | 6 | 39 | 2 |
| 10 | 2016 | April | The Australian Financial System | Financial Stability Review – April 2016 | RBA | Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. | 1_frs | G1 | 1 | 2 | 10 | 1_2_10 | Australian banks using the IRB approach to credit risk have been required to disclose their leverage ratio from mid 2015. The leverage ratio is a non-risk-based measure of a bank’s Tier 1 capital relative to its total exposures, and is intended to be a backstop to the risk-based capital requirements. The leverage ratio framework is yet to be finalised internationally, although the Basel Committee’s governing body agreed the minimum requirement should be 3 per cent. The Basel Committee is expected to make final adjustments to the measure by the end of 2016, with a view to establishing the requirement from January 2018. Each of the Australian banks required to disclose the measure reported a leverage ratio close to 5 per cent at December 2015, well above the minimum. | 121 | 5 | 228 | 31 | 16.08271 | 22.86043 | 4 | 10 | 19 | 10 | 1 | 19 | 12 | 9 | 0 | 1 | 35 | 5 | 0 | 2 | 4 | 0 | 11 | 6 | 3 | 3 | 5 | 1 | 4 | 0 | 7 | 0 | 2 | 0 | 0 | 0 | 0 | 0.77 | 14.62 | 9.23 | 6.92 | 0.00 | 0.77 | 26.92 | 3.85 | 0.00 | 1.54 | 3.08 | 0.00 | 8.46 | 4.62 | 2.31 | 2.31 | 3.85 | 0.77 | 3.08 | 0.00 | 5.38 | 0.00 | 1.54 | 0.00 | 0.00 | 0.00 | 0.00 | australian | NN | australian | NN | banks | NNS | using | VBG | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 4 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 6 | 2 | 2 | 0 | 6 | 0 | 0 | 2 | 9 | 4 | 2 | 0 | 4 | 0 | 1 | 4 | 40 | 15 | 13 | 2 | 28 | 0 |
The session information for this program is:
sessionInfo()## R version 4.0.3 (2020-10-10)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 17763)
##
## Matrix products: default
##
## locale:
## [1] LC_COLLATE=English_Australia.1252 LC_CTYPE=English_Australia.1252
## [3] LC_MONETARY=English_Australia.1252 LC_NUMERIC=C
## [5] LC_TIME=English_Australia.1252
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] plyr_1.8.6 openNLP_0.2-7 sylcount_0.2-2
## [4] rvest_1.0.0 xml2_1.3.2 forcats_0.5.1
## [7] purrr_0.3.4 readr_1.4.0 tidyverse_1.3.1
## [10] quanteda_3.0.0 textclean_0.9.3 kableExtra_1.3.4
## [13] tidyr_1.1.3 MASS_7.3-53 data.table_1.14.0
## [16] lubridate_1.7.10 tibble_3.1.1 zoo_1.8-9
## [19] reshape2_1.4.4 stringi_1.5.3 reshape_0.8.8
## [22] qdap_2.4.3 qdapTools_1.3.5 qdapRegex_0.7.2
## [25] qdapDictionaries_1.0.7 stargazer_5.2.2 e1071_1.7-6
## [28] stringr_1.4.0 tm_0.7-8 NLP_0.2-1
## [31] wordcloud_2.6 RColorBrewer_1.1-2 ggplot2_3.3.3
## [34] tidytext_0.3.1 dplyr_1.0.6
##
## loaded via a namespace (and not attached):
## [1] fs_1.5.0 bitops_1.0-7 webshot_0.5.2
## [4] httr_1.4.2 SnowballC_0.7.0 tools_4.0.3
## [7] backports_1.2.1 utf8_1.2.1 R6_2.5.0
## [10] DBI_1.1.1 colorspace_2.0-1 openNLPdata_1.5.3-4
## [13] withr_2.4.2 tidyselect_1.1.1 gridExtra_2.3
## [16] compiler_4.0.3 chron_2.3-56 cli_2.5.0
## [19] slam_0.1-48 scales_1.1.1 proxy_0.4-25
## [22] systemfonts_1.0.1 digest_0.6.27 rmarkdown_2.8
## [25] svglite_2.0.0 pkgconfig_2.0.3 htmltools_0.5.1.1
## [28] plotrix_3.8-1 highr_0.9 dbplyr_2.1.1
## [31] readxl_1.3.1 rlang_0.4.11 rstudioapi_0.13
## [34] generics_0.1.0 jsonlite_1.7.2 zip_2.1.1
## [37] tokenizers_0.2.1 RCurl_1.98-1.3 magrittr_2.0.1
## [40] Matrix_1.2-18 Rcpp_1.0.6 munsell_0.5.0
## [43] fansi_0.4.2 lifecycle_1.0.0 yaml_2.2.1
## [46] grid_4.0.3 parallel_4.0.3 gender_0.5.4
## [49] crayon_1.4.1 lattice_0.20-41 haven_2.4.1
## [52] hms_1.0.0 knitr_1.33 venneuler_1.1-0
## [55] pillar_1.6.0 igraph_1.2.6 stopwords_2.2
## [58] fastmatch_1.1-0 reprex_2.0.0 XML_3.99-0.6
## [61] glue_1.4.2 evaluate_0.14 modelr_0.1.8
## [64] RcppParallel_5.1.4 vctrs_0.3.8 cellranger_1.1.0
## [67] gtable_0.3.0 assertthat_0.2.1 xfun_0.22
## [70] openxlsx_4.2.3 broom_0.7.6 janeaustenr_0.1.5
## [73] class_7.3-17 viridisLite_0.4.0 rJava_1.0-4
## [76] ellipsis_0.3.2