arm Linux 中断管理机制
關鍵詞:GIC、IAR、EOI、SGI/PPI/SPI、中斷映射、中斷異常向量、中斷上下文、內核中斷線程、中斷注冊。
1.1 ARM支持中斷類型
ARM GIC-v2支持三種類型的中斷:
SGI:軟件觸發中斷(Software Generated Interrupt),通常用于多核間通訊,最多支持16個SGI中斷,硬件中斷號從ID0~ID15。SGI通常在Linux內核中被用作IPI中斷(inter-processor interrupts),并會送達到系統指定的CPU上。
PPI:私有外設中斷(Private Peripheral Interrupt),是每個CPU私有的中斷。最多支持16個PPI中斷,硬件中斷號從ID16~ID31。PPI通常會送達到指定的CPU上,應用場景有CPU本地時鐘。
SPI:公用外設中斷(Shared Peripheral Interrupt),最多可以支持988個外設中斷,硬件中斷號從ID32~ID1019。
1.2 GIC檢測中斷流程
GIC(Generic Interrupt Controller,通用中斷控制器)主要由兩部分組成,分別是仲裁單元(Distributor)和CPU接口模塊。
GIC仲裁單元為每一個中斷維護一個狀態機,分別是:inactive、pending、active and pending、active。
下面是來自IHI0048B GIC-V2規格書3.2.4 Interrupt handling state machine截圖:
?
GIC檢測中斷流程如下:
(1) 當GIC檢測到一個中斷發生時,會將該中斷標記為pending狀態(A1)。
(2) 對處于pending狀態的中斷,仲裁單元回確定目標CPU,將中斷請求發送到這個CPU上。
(3) 對于每個CPU,仲裁單元會從眾多pending狀態的中斷中選擇一個優先級最高的中斷,發送到目標CPU的CPU Interface模塊上。
(4) CPU Interface會決定這個中斷是否可以發送給CPU。如果該終端優先級滿足要求,GIC會發生一個中斷信號給該CPU。
(5) 當一個CPU進入中斷異常后,會去讀取GICC_IAR寄存器來響應該中斷(一般是Linux內核的中斷處理程序來讀寄存器)。寄存器會返回硬件中斷號(hardware interrupt ID),對于SGI中斷來說是返回源CPU的ID。
當GIC感知到軟件讀取了該寄存器后,又分為如下情況:
* 如果該中斷源是pending狀態,那么轉改將變成active。(C)
* 如果該中斷又重新產生,那么pending狀態變成active and pending。(D)
* 如果該中斷是active狀態,現在變成active and pending。(A2)
(6) 當處理器完成中斷服務,必須發送一個完成信號EOI(End Of Interrupt)給GIC控制器。軟件寫GICC_EOIR寄存器,狀態變成inactive。(E1)
補充:
(7) 對于level triggered類型中斷來說,當觸發電平消失,狀態從active and pending變成active。(B2)
常用路徑是A1->D->B2->E1。
1.2.1 GIC中斷搶占
GIC中斷控制器支持中斷優先級搶占,一個高優先級中斷可以搶占一個低優先級且處于active狀態的中斷,即GIC仲裁單元會記錄和比較當前優先級最高的pending狀態,然后去搶占當前中斷,并且發送這個最高優先級的中斷請求給CPU,CPU應答了高優先級中斷,暫停低優先級中斷服務,進而去處理高優先級中斷。
GIC會將pending狀態優先級最高的中斷請求發送給CPU。
1.2.2 Linux對中斷搶占處理
從GIC角度看,GIC會發送高優先級中斷請求給CPU。
但是目前CPU處于關中斷狀態,需要等低優先級中斷處理完畢,直到發送EOI給GIC。
然后CPU才會響應pending狀態中優先級最高的中斷進行處理。
所以Linux下:
1. 高優先級中斷無法搶占正在執行的低優先級中斷。
2.同處于pending狀態的中斷,優先響應高優先級中斷進行處理。
1.3 GIC中斷時序
?
借助GIC-400 Figure B-1?Signaling physical interrupts 理解GIC內部工作原理。
M和N都是SPI類型的外設中斷,且通過FIQ來處理,高電平觸發,N的優先級比M高,他們的目標CPU相同。
(1) T1時刻:GIC的總裁單元檢測到中斷M的電平變化。
(2) T2時刻:仲裁單元設置中斷M的狀態為pending。
(3) T17時刻:CPU Interface模塊會拉低nFIQCPU[n]信號。在中斷M的狀態變成pending后,大概需要15個時鐘周期后會拉低nFIQCPU[n]信號來向CPU報告中斷請求(assertion)。仲裁單元需要這些時間來計算哪個是pending狀態下優先級最高的中斷。
(4) T42時刻:仲裁單元檢測到另外一個優先級更高的中斷N。
(5) T43時刻:仲裁單元用中斷N替換中斷M為當前pending狀態下優先級最高的中斷,并設置中斷N為pending狀態。
(6) T58時刻:經過tph個時鐘后,CPU Interface拉低你FIOCPU[n]信號來通知CPU。因為此信號在T17時刻已經被拉低,CPU Interface模塊會更新GICC_IAR寄存器的Interrupt ID域,該域的值變成中斷N的硬件中斷號。
(7) T61~T131時刻:Linux對中斷N的服務程序--------------------------------------------------------------中斷服務程序處理段,從GICC_IAR開始到GICC_EOIR結束。
T61時刻:CPU(Linux中斷服務例程)讀取GICC_IAR寄存器,即軟件響應了中斷N。這時仲裁單元把中斷N的狀態從pending變成active and pending。讀取GICC_IAR
T64時刻:在中斷N被Linux相應3個時鐘內,CPU Interface模塊完成對nFIQCPU[n]信號的deasserts,即拉高nFIQCPU[n]信號。
T126時刻:外設也deassert了該中斷N。
T128時刻:仲裁單元移出了中斷N的pending狀態。
T131時刻:Linux服務程序把中斷N的硬件ID號寫入GICC_EOIR寄存器來完成中斷N的全部處理過程。寫GICC_EOIR
(8) T146時刻:在向GICC_EOIR寄存器寫入中斷N中斷號后的tph個時鐘后,仲裁單元會選擇下一個最高優先級中斷,即中斷M,發送中斷請求給CPU Interface模塊。CPU Interface會拉低nFIQCPU[n]信號來向CPU報告外設M的中斷請求。
(9) T211時刻:Linux中斷服務程序讀取GICC_IAR寄存器來響應中斷,仲裁單元設置中斷M的狀態為active and pending。
(10) T214時刻:在CPU響應中斷后的3個時鐘內,CPU Interface模塊拉高nFIOCPU[n]信號來完成deassert動作。
那么GICC_IAR和GICC_EOIR分別在Linux什么地方觸發的呢?
1.4 Cortex A15 A7實例
2. 硬件中斷號和Linux中斷號的映射
2.1 硬件中斷號:一個串口中斷實例
2.2 中斷控制器初始化
DTS中GIC定義于arch/arm/boot/dts/vexpress-v2p-ca15_a7.dts:
gic: interrupt-controller@2c001000 {compatible = "arm,cortex-a15-gic", "arm,cortex-a9-gic";------------------此設備的標識符是"arm,cortex-a15-gic"#interrupt-cells = <3>;#address-cells = <0>;interrupt-controller;----------------------------------------------------表示此設備是一個中斷控制器reg = <0 0x2c001000 0 0x1000>,<0 0x2c002000 0 0x1000>,<0 0x2c004000 0 0x2000>,<0 0x2c006000 0 0x2000>;interrupts = <1 9 0xf04>;};struct irq_domain用于描述一個中斷控制器。
GIC中斷控制器在初始化時解析DTS信息中定義了幾個GIC控制器,每個GIC控制器注冊一個struct irq_domain數據結構。
struct irq_domain {struct list_head link;-------------------------用于將irq_domain連接到全局鏈表irq_domain_list中。const char *name;------------------------------中斷控制器名稱const struct irq_domain_ops *ops;--------------irq domain映射操作使用的方法集合void *host_data;unsigned int flags;/* Optional data */struct device_node *of_node;------------------對應中斷控制器的device nodestruct irq_domain_chip_generic *gc; #ifdef CONFIG_IRQ_DOMAIN_HIERARCHYstruct irq_domain *parent; #endif/* reverse map data. The linear map gets appended to the irq_domain */irq_hw_number_t hwirq_max;--------------------該irq domain支持中斷數量的最大值。unsigned int revmap_direct_max_irq;unsigned int revmap_size;---------------------線性映射的大小struct radix_tree_root revmap_tree;-----------Radix Tree映射的根節點unsigned int linear_revmap[];-----------------線性映射用到的lookup table }struct irq_domain_ops定義了irq_domain方法集合,xlate從intspec中解析出硬件中斷號和中斷類型,intspec[0]和intspec[1]決定中斷號,intspec[2]決定中斷類型。
struct irq_domain_ops {int (*match)(struct irq_domain *d, struct device_node *node);int (*map)(struct irq_domain *d, unsigned int virq, irq_hw_number_t hw);void (*unmap)(struct irq_domain *d, unsigned int virq);int (*xlate)(struct irq_domain *d, struct device_node *node,const u32 *intspec, unsigned int intsize,unsigned long *out_hwirq, unsigned int *out_type);#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY/* extended V2 interfaces to support hierarchy irq_domains */int (*alloc)(struct irq_domain *d, unsigned int virq,unsigned int nr_irqs, void *arg);void (*free)(struct irq_domain *d, unsigned int virq,unsigned int nr_irqs);void (*activate)(struct irq_domain *d, struct irq_data *irq_data);void (*deactivate)(struct irq_domain *d, struct irq_data *irq_data); #endif };static const struct irq_domain_ops gic_irq_domain_hierarchy_ops = {.xlate = gic_irq_domain_xlate,.alloc = gic_irq_domain_alloc,.free = irq_domain_free_irqs_top, };static int gic_irq_domain_xlate(struct irq_domain *d,struct device_node *controller,const u32 *intspec, unsigned int intsize,unsigned long *out_hwirq, unsigned int *out_type) { .../* Get the interrupt number and add 16 to skip over SGIs */*out_hwirq = intspec[1] + 16;--------------------------------------首先+16跳過SGI類型中斷/* For SPIs, we need to add 16 more to get the GIC irq ID number */if (!intspec[0]) {-------------------------------------------------如果是SPI類型中斷,還需要+16,跳過PPI類型中斷。ret = gic_routable_irq_domain_ops->xlate(d, controller,intspec,intsize,out_hwirq,out_type);if (IS_ERR_VALUE(ret))return ret;}*out_type = intspec[2] & IRQ_TYPE_SENSE_MASK;---------------------中斷觸發類型,包括四種上升沿、下降沿、高電平、低電平。return ret; }static int gic_irq_domain_alloc(struct irq_domain *domain, unsigned int virq,unsigned int nr_irqs, void *arg) {int i, ret;irq_hw_number_t hwirq;unsigned int type = IRQ_TYPE_NONE;struct of_phandle_args *irq_data = arg;ret = gic_irq_domain_xlate(domain, irq_data->np, irq_data->args,irq_data->args_count, &hwirq, &type);---------------首先根據args翻譯出硬件中斷號和中斷類型。if (ret)return ret;for (i = 0; i < nr_irqs; i++)gic_irq_domain_map(domain, virq + i, hwirq + i);---------------執行軟硬件的映射,并且根據中斷類型設置struct irq_desc->handle_irq處理函數。return 0; }void irq_domain_free_irqs_top(struct irq_domain *domain, unsigned int virq,unsigned int nr_irqs) {int i;for (i = 0; i < nr_irqs; i++) {irq_set_handler_data(virq + i, NULL);irq_set_handler(virq + i, NULL);}irq_domain_free_irqs_common(domain, virq, nr_irqs); }針對SPI類型中斷,需要進行+16位移。?
static int gic_routable_irq_domain_xlate(struct irq_domain *d,struct device_node *controller,const u32 *intspec, unsigned int intsize,unsigned long *out_hwirq,unsigned int *out_type) {*out_hwirq += 16;return 0; }gic_irq_domain_map()入參有struct irq_domain和軟硬件中斷號,主要分SGI/PPI一組,SPI一組。
主要工作由irq_domain_set_info()處理,irq_domain_set_hwirq_and_chip()通過Linux中斷號獲取struct irq_data數據結構,設置關聯硬件中斷號和struct irq_chip gic_chip關聯。
__irq_set_handler()設置中斷描述符irq_desc->handler_irq回調函數,對SPI類型來說就是handle_fasteoi_irq()。
static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,irq_hw_number_t hw) {if (hw < 32) {irq_set_percpu_devid(irq);-------------------------------PerCPU類型的中斷有自己的特殊flag。irq_domain_set_info(d, irq, hw, &gic_chip, d->host_data,handle_percpu_devid_irq, NULL, NULL);set_irq_flags(irq, IRQF_VALID | IRQF_NOAUTOEN);} else {irq_domain_set_info(d, irq, hw, &gic_chip, d->host_data,handle_fasteoi_irq, NULL, NULL);set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);gic_routable_irq_domain_ops->map(d, irq, hw);}return 0; }void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,irq_hw_number_t hwirq, struct irq_chip *chip,void *chip_data, irq_flow_handler_t handler,void *handler_data, const char *handler_name) {irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);__irq_set_handler(virq, handler, 0, handler_name);irq_set_handler_data(virq, handler_data); }int irq_domain_set_hwirq_and_chip(struct irq_domain *domain, unsigned int virq,irq_hw_number_t hwirq, struct irq_chip *chip,void *chip_data) {struct irq_data *irq_data = irq_domain_get_irq_data(domain, virq);if (!irq_data)return -ENOENT;irq_data->hwirq = hwirq;irq_data->chip = chip ? chip : &no_irq_chip;irq_data->chip_data = chip_data;return 0; }void __irq_set_handler(unsigned int irq, irq_flow_handler_t handle, int is_chained,const char *name) {unsigned long flags;struct irq_desc *desc = irq_get_desc_buslock(irq, &flags, 0); ...desc->handle_irq = handle;--------------------irq_desc->handler_irq和name賦值。desc->name = name; ... }drivers/irqchip/irq-gic.c定義了"arm,cortex-a15-gic"的處理函數gic_of_init,gic_of_init是GIC控制器的初始化函數。
IRQCHIP_DECLARE(cortex_a15_gic, "arm,cortex-a15-gic", gic_of_init);static int gic_cnt __initdata;static int __init gic_of_init(struct device_node *node, struct device_node *parent) { ...gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset, node);if (!gic_cnt)gic_init_physaddr(node);if (parent) {irq = irq_of_parse_and_map(node, 0);gic_cascade_irq(gic_cnt, irq);}if (IS_ENABLED(CONFIG_ARM_GIC_V2M))gicv2m_of_init(node, gic_data[gic_cnt].domain);gic_cnt++;return 0; }gic_init_bases的gic_nr是GIC控制器的序號,主要調用irq_domain_add_linear()分配并函數注冊一個irq_domain。
void __init gic_init_bases(unsigned int gic_nr, int irq_start,void __iomem *dist_base, void __iomem *cpu_base,u32 percpu_offset, struct device_node *node) {irq_hw_number_t hwirq_base;struct gic_chip_data *gic;int gic_irqs, irq_base, i;int nr_routable_irqs;BUG_ON(gic_nr >= MAX_GIC_NR);---------------------------gic_nr不超過系統規定的MAX_GIC_NRgic = &gic_data[gic_nr];--------------------------------struct gic_chip_data類型的全局變量gic_data,序號是GIC控制器序號 ... /** Initialize the CPU interface map to all CPUs.* It will be refined as each CPU probes its ID.*/for (i = 0; i < NR_GIC_CPU_IF; i++)gic_cpu_map[i] = 0xff;/** Find out how many interrupts are supported.* The GIC only supports up to 1020 interrupt sources.*/gic_irqs = readl_relaxed(gic_data_dist_base(gic) + GIC_DIST_CTR) & 0x1f;------------計算GIC控制器最多支持的中斷源個數gic_irqs = (gic_irqs + 1) * 32;if (gic_irqs > 1020)-------------------------------------------------------GIC支持的最大中斷數據,此處為1020gic_irqs = 1020;gic->gic_irqs = gic_irqs;if (node) { /* DT case */const struct irq_domain_ops *ops = &gic_irq_domain_hierarchy_ops;--------------GICv2的struct irq_domain_ops ...gic->domain = irq_domain_add_linear(node, gic_irqs, ops, gic);-----------------注冊irq_domain,操作函數使用gic_irq_domain_hierarchy_ops} else { /* Non-DT case */ ...}if (WARN_ON(!gic->domain))return;if (gic_nr == 0) { #ifdef CONFIG_SMPset_smp_cross_call(gic_raise_softirq);register_cpu_notifier(&gic_cpu_notifier); #endifset_handle_irq(gic_handle_irq);-------在irq_handler中調用handle_arch_irq,這里將handle_arch_irq指向gic_handle_irq,實現了平臺中斷和具體GIC中斷的關聯。}gic_chip.flags |= gic_arch_extn.flags;gic_dist_init(gic);----------------------GIC Distributer部分初始化gic_cpu_init(gic);-----------------------GIC CPU Interface部分初始化gic_pm_init(gic);------------------------GIC PM相關初始化 }irq_domain_add_linear()->__irq_domain_add()分配并初始化struct irq_domain。
struct irq_domain *__irq_domain_add(struct device_node *of_node, int size,irq_hw_number_t hwirq_max, int direct_max,const struct irq_domain_ops *ops,void *host_data) {struct irq_domain *domain;domain = kzalloc_node(sizeof(*domain) + (sizeof(unsigned int) * size),GFP_KERNEL, of_node_to_nid(of_node));-------------domain大小為struct irq_domain加上gic_irqs個unsigned int。if (WARN_ON(!domain))return NULL;/* Fill structure */INIT_RADIX_TREE(&domain->revmap_tree, GFP_KERNEL);domain->ops = ops;domain->host_data = host_data;domain->of_node = of_node_get(of_node);domain->hwirq_max = hwirq_max;domain->revmap_size = size;domain->revmap_direct_max_irq = direct_max;irq_domain_check_hierarchy(domain);mutex_lock(&irq_domain_mutex);list_add(&domain->link, &irq_domain_list);----------------------將創建好的struct irq_domain加入全局鏈表irq_domain_list。mutex_unlock(&irq_domain_mutex);pr_debug("Added domain %s\n", domain->name);return domain; }2.3 系統初始化之中斷號映射
上一小節是中斷控制器GIC的初始化,下面看看一個硬件中斷是如何映射到Linux空間的中斷的。
customize_machine()是arch_initcall階段調用,很靠前。
customize_machine ->of_platform_populate ->of_platform_bus_create ->of_amba_device_create ->of_amba_device_create
下面結合dtsi文件看看來龍去脈,arch/arm/boot/dts/vexpress-v2m.dtsi。
/dts-v1/;/ {model = "V2P-CA9";arm,hbi = <0x191>;arm,vexpress,site = <0xf>;compatible = "arm,vexpress,v2p-ca9", "arm,vexpress";interrupt-parent = <&gic>;#address-cells = <1>;#size-cells = <1>; ...gic: interrupt-controller@1e001000 {compatible = "arm,cortex-a9-gic";#interrupt-cells = <3>;#address-cells = <0>;interrupt-controller;reg = <0x1e001000 0x1000>,<0x1e000100 0x100>;}; ...smb {compatible = "simple-bus";#address-cells = <2>;#size-cells = <1>;ranges = <0 0 0x40000000 0x04000000>,<1 0 0x44000000 0x04000000>,<2 0 0x48000000 0x04000000>,<3 0 0x4c000000 0x04000000>,<7 0 0x10000000 0x00020000>;#interrupt-cells = <1>;interrupt-map-mask = <0 0 63>;interrupt-map = <0 0 0 &gic 0 0 4>,<0 0 1 &gic 0 1 4>, ... /include/ "vexpress-v2m.dtsi"}; }vexpress-v2m.dtsi文件:motherboard {model = "V2M-P1";arm,hbi = <0x190>;arm,vexpress,site = <0>;compatible = "arm,vexpress,v2m-p1", "simple-bus";#address-cells = <2>; /* SMB chipselect number and offset */#size-cells = <1>;#interrupt-cells = <1>;ranges; ...iofpga@7,00000000 {compatible = "arm,amba-bus", "simple-bus";#address-cells = <1>;#size-cells = <1>;ranges = <0 7 0 0x20000>; ...v2m_serial0: uart@09000 {compatible = "arm,pl011", "arm,primecell";reg = <0x09000 0x1000>;interrupts = <5>;clocks = <&v2m_oscclk2>, <&smbclk>;clock-names = "uartclk", "apb_pclk";}; ...};}這里首先從根目錄下查找"simple-bus",從上面可以看出指向smb設備。
smb設備包含vexpress-v2m.dtsi文件,然后在of_platform_bus_create()中遍歷所有設備。
const struct of_device_id of_default_bus_match_table[] = {{ .compatible = "simple-bus", }, #ifdef CONFIG_ARM_AMBA{ .compatible = "arm,amba-bus", }, #endif /* CONFIG_ARM_AMBA */{} /* Empty terminated list */ };static int __init customize_machine(void) { ...of_platform_populate(NULL, of_default_bus_match_table,-----------------找到匹配"simple-bus"的設備,這里指向smb。NULL, NULL); ... }int of_platform_populate(struct device_node *root,const struct of_device_id *matches,const struct of_dev_auxdata *lookup,struct device *parent) { ...for_each_child_of_node(root, child) {rc = of_platform_bus_create(child, matches, lookup, parent, true);-----這里的root指向根目錄,即"/"。if (rc)break;} ... }static int of_platform_bus_create(struct device_node *bus,const struct of_device_id *matches,const struct of_dev_auxdata *lookup,struct device *parent, bool strict) {const struct of_dev_auxdata *auxdata;struct device_node *child;struct platform_device *dev;const char *bus_id = NULL;void *platform_data = NULL;int rc = 0;/* Make sure it has a compatible property */if (strict && (!of_get_property(bus, "compatible", NULL))) {pr_debug("%s() - skipping %s, no compatible prop\n",__func__, bus->full_name);return 0;}auxdata = of_dev_lookup(lookup, bus);if (auxdata) {bus_id = auxdata->name;platform_data = auxdata->platform_data;}if (of_device_is_compatible(bus, "arm,primecell")) {------當遇到匹配"arm,primecell"設備,創建amba設備。在ofpga@7,00000000中創建uart@09000設備。/** Don't return an error here to keep compatibility with older* device tree files.*/of_amba_device_create(bus, bus_id, platform_data, parent);return 0;}dev = of_platform_device_create_pdata(bus, bus_id, platform_data, parent);if (!dev || !of_match_node(matches, bus))return 0;for_each_child_of_node(bus, child) {----------------遍歷smb下的所有"simple-bus"設備,這里可以嵌套幾層。從smb->motherboard->iofpga@7,00000000。pr_debug(" create child: %s\n", child->full_name);rc = of_platform_bus_create(child, matches, lookup, &dev->dev, strict);if (rc) {of_node_put(child);break;}}of_node_set_flag(bus, OF_POPULATED_BUS);return rc; }of_amba_device_create創建ARM AMBA類型設備,其中中斷部分交給irq_of_parse_and_map()處理。
static struct amba_device *of_amba_device_create(struct device_node *node,const char *bus_id,void *platform_data,struct device *parent) { .../* Decode the IRQs and address ranges */for (i = 0; i < AMBA_NR_IRQS; i++)dev->irq[i] = irq_of_parse_and_map(node, i); ... }以uart@09000為例,irq_of_parse_and_map中的of_irq_parse_one()解析設備中的"interrupts"、"regs"等參數,參數放入struct of_phandle_args中,oirq->args[1]中存放中斷號5,oirq->np存放struct device_node。
irq_create_of_mapping()建立硬件中斷號到Linux中斷號的映射。
irq_create_of_mapping主要調用如下,主要工作交給__irq_domain_alloc_irqs()進行處理。
irq_create_of_mapping ->domain->ops->xlate--------------------------------- ->irq_find_mapping ->irq_domain_alloc_irqs ->__irq_domain_alloc_irqs ->irq_domain_alloc_descs ->irq_domain_alloc_irq_data ->irq_domain_alloc_irqs_recursive ->gic_irq_domain_alloc ->gic_irq_domain_map-----------------------進行硬件中斷號和軟件中斷號的映射 ->gic_irq_domain_set_info----------------設置重要參數到中斷描述符中 ->irq_domain_insert_irq
unsigned int irq_of_parse_and_map(struct device_node *dev, int index) {struct of_phandle_args oirq;if (of_irq_parse_one(dev, index, &oirq))return 0;return irq_create_of_mapping(&oirq); }unsigned int irq_create_of_mapping(struct of_phandle_args *irq_data) {struct irq_domain *domain;irq_hw_number_t hwirq;unsigned int type = IRQ_TYPE_NONE;int virq;domain = irq_data->np ? irq_find_host(irq_data->np) : irq_default_domain;---找到設備所屬的struct irq_domain結構體。 .../* If domain has no translation, then we assume interrupt line */if (domain->ops->xlate == NULL)hwirq = irq_data->args[0];else {if (domain->ops->xlate(domain, irq_data->np, irq_data->args,-------調用gic_irq_domain_xlate()函數進行硬件中斷號到Linux中斷號的轉換。irq_data->args_count, &hwirq, &type))return 0;}if (irq_domain_is_hierarchy(domain)) {-------------------------可以分層掛載/** If we've already configured this interrupt,* don't do it again, or hell will break loose.*/virq = irq_find_mapping(domain, hwirq);-------------------從已有的linear_revmap中尋找Linux中斷號。if (virq)return virq;virq = irq_domain_alloc_irqs(domain, 1, NUMA_NO_NODE, irq_data);---------如果沒有找到,重新分配中斷映射。參數1表示每次只分配一個中斷。if (virq <= 0)return 0;} else { ...}/* Set type if specified and different than the current one */if (type != IRQ_TYPE_NONE &&type != irq_get_trigger_type(virq))irq_set_irq_type(virq, type);-----------------------------設置中斷觸發類型return virq; }struct irq_desc定義了中斷描述符,irq_desc[]數組定義了NR_IRQS個中斷描述符,數組下標表示IRQ中斷號,通過IRQ中斷號可以找到對應中斷描述符。
struct irq_desc內置了struct irq_data結構體,struct irq_data的irq和hwirq分別對應軟件中斷號和硬件中斷號。通過這兩個成員,可以將硬件中斷號和軟件中斷號映射起來。
struct irq_chip定義了中斷控制器底層操作相關的方法集合。
struct irq_desc {struct irq_data irq_data;unsigned int __percpu *kstat_irqs;irq_flow_handler_t handle_irq;-----------------根據中斷號分類,不同類型中斷的處理handle。0~31對應handle_percpu_devid_irq;32~對應handle_fasteoi_irq。 #ifdef CONFIG_IRQ_PREFLOW_FASTEOIirq_preflow_handler_t preflow_handler; #endifstruct irqaction *action; /* IRQ action list */unsigned int status_use_accessors;unsigned int core_internal_state__do_not_mess_with_it;unsigned int depth; /* nested irq disables */unsigned int wake_depth; /* nested wake enables */unsigned int irq_count; /* For detecting broken IRQs */unsigned long last_unhandled; /* Aging timer for unhandled count */unsigned int irqs_unhandled;atomic_t threads_handled;int threads_handled_last;raw_spinlock_t lock;struct cpumask *percpu_enabled; #ifdef CONFIG_SMPconst struct cpumask *affinity_hint;struct irq_affinity_notify *affinity_notify; #ifdef CONFIG_GENERIC_PENDING_IRQcpumask_var_t pending_mask; #endif #endifunsigned long threads_oneshot;-------------是一個位圖,每個比特位代表正在處理的共享oneshot類型中斷的中斷線程。atomic_t threads_active;-------------------表示正在運行的中斷線程個數wait_queue_head_t wait_for_threads; #ifdef CONFIG_PM_SLEEPunsigned int nr_actions;unsigned int no_suspend_depth;unsigned int cond_suspend_depth;unsigned int force_resume_depth; #endif #ifdef CONFIG_PROC_FSstruct proc_dir_entry *dir; #endifint parent_irq;struct module *owner;const char *name; }struct irq_data {u32 mask;unsigned int irq;-----------------Linux軟件中斷號unsigned long hwirq;--------------硬件中斷號unsigned int node;unsigned int state_use_accessors;struct irq_chip *chip;struct irq_domain *domain; #ifdef CONFIG_IRQ_DOMAIN_HIERARCHYstruct irq_data *parent_data; #endifvoid *handler_data;void *chip_data;struct msi_desc *msi_desc;cpumask_var_t affinity; }struct irq_chip {const char *name;unsigned int (*irq_startup)(struct irq_data *data);-------------初始化中斷void (*irq_shutdown)(struct irq_data *data);----------------結束中斷void (*irq_enable)(struct irq_data *data);------------------使能中斷void (*irq_disable)(struct irq_data *data);-----------------關閉中斷void (*irq_ack)(struct irq_data *data);---------------------應答中斷void (*irq_mask)(struct irq_data *data);--------------------屏蔽中斷void (*irq_mask_ack)(struct irq_data *data);----------------應答并屏蔽中斷void (*irq_unmask)(struct irq_data *data);------------------解除中斷屏蔽void (*irq_eoi)(struct irq_data *data);---------------------發送EOI信號,表示硬件中斷處理已經完成。int (*irq_set_affinity)(struct irq_data *data, const struct cpumask *dest, bool force);--------綁定中斷到某個CPUint (*irq_retrigger)(struct irq_data *data);----------------重新發送中斷到CPUint (*irq_set_type)(struct irq_data *data, unsigned int flow_type);----------------------------設置觸發類型int (*irq_set_wake)(struct irq_data *data, unsigned int on);-----------------------------------使能/關閉中斷在電源管理中的喚醒功能。void (*irq_bus_lock)(struct irq_data *data);void (*irq_bus_sync_unlock)(struct irq_data *data);void (*irq_cpu_online)(struct irq_data *data);void (*irq_cpu_offline)(struct irq_data *data);void (*irq_suspend)(struct irq_data *data);void (*irq_resume)(struct irq_data *data);void (*irq_pm_shutdown)(struct irq_data *data); ...unsigned long flags; }gic_chip是特定中斷控制器的硬件操作函數集,對于GICv2有屏蔽/去屏蔽、EOI、設置中斷觸發類型、以及設置或者當前芯片狀態。
static const struct irq_chip gic_chip = {.irq_mask = gic_mask_irq,.irq_unmask = gic_unmask_irq,.irq_eoi = gic_eoi_irq,.irq_set_type = gic_set_type,.irq_get_irqchip_state = gic_irq_get_irqchip_state,.irq_set_irqchip_state = gic_irq_set_irqchip_state,.flags = IRQCHIP_SET_TYPE_MASKED |IRQCHIP_SKIP_SET_WAKE |IRQCHIP_MASK_ON_SUSPEND, };static void gic_mask_irq(struct irq_data *d) {gic_poke_irq(d, GIC_DIST_ENABLE_CLEAR); }static void gic_unmask_irq(struct irq_data *d) {gic_poke_irq(d, GIC_DIST_ENABLE_SET); }static void gic_eoi_irq(struct irq_data *d) {writel_relaxed(gic_irq(d), gic_cpu_base(d) + GIC_CPU_EOI); }static int gic_set_type(struct irq_data *d, unsigned int type) {void __iomem *base = gic_dist_base(d);unsigned int gicirq = gic_irq(d);/* Interrupt configuration for SGIs can't be changed */if (gicirq < 16)return -EINVAL;/* SPIs have restrictions on the supported types */if (gicirq >= 32 && type != IRQ_TYPE_LEVEL_HIGH &&type != IRQ_TYPE_EDGE_RISING)return -EINVAL;return gic_configure_irq(gicirq, type, base, NULL); }static int gic_irq_set_irqchip_state(struct irq_data *d,enum irqchip_irq_state which, bool val) {u32 reg;switch (which) {case IRQCHIP_STATE_PENDING:reg = val ? GIC_DIST_PENDING_SET : GIC_DIST_PENDING_CLEAR;break;case IRQCHIP_STATE_ACTIVE:reg = val ? GIC_DIST_ACTIVE_SET : GIC_DIST_ACTIVE_CLEAR;break;case IRQCHIP_STATE_MASKED:reg = val ? GIC_DIST_ENABLE_CLEAR : GIC_DIST_ENABLE_SET;break;default:return -EINVAL;}gic_poke_irq(d, reg);return 0; }static int gic_irq_get_irqchip_state(struct irq_data *d,enum irqchip_irq_state which, bool *val) {switch (which) {case IRQCHIP_STATE_PENDING:*val = gic_peek_irq(d, GIC_DIST_PENDING_SET);break;case IRQCHIP_STATE_ACTIVE:*val = gic_peek_irq(d, GIC_DIST_ACTIVE_SET);break;case IRQCHIP_STATE_MASKED:*val = !gic_peek_irq(d, GIC_DIST_ENABLE_SET);break;default:return -EINVAL;}return 0; }irq_domain_alloc_irqs()調用__irq_domain_alloc_irqs()進行struct irq_desc、struct irq_data以及中斷映射的處理。
這里的參數nr_irqs一般為1,每次只處理一個中斷。
irq_domain_alloc_descs()->irq_alloc_descs()->__irq_alloc_descs()進行struct irq_desc的分配,返回的參數是Linux中斷號。
int __irq_domain_alloc_irqs(struct irq_domain *domain, int irq_base,unsigned int nr_irqs, int node, void *arg,bool realloc) { ...if (realloc && irq_base >= 0) {virq = irq_base;} else {virq = irq_domain_alloc_descs(irq_base, nr_irqs, 0, node);-------從allocated_irqs位圖中查找第一個nr_irqs個空閑的比特位,最終調用__irq_alloc_descs。if (virq < 0) {pr_debug("cannot allocate IRQ(base %d, count %d)\n",irq_base, nr_irqs);return virq;}}if (irq_domain_alloc_irq_data(domain, virq, nr_irqs)) {--------------分配struct irq_data數據結構。pr_debug("cannot allocate memory for IRQ%d\n", virq);ret = -ENOMEM;goto out_free_desc;}mutex_lock(&irq_domain_mutex);ret = irq_domain_alloc_irqs_recursive(domain, virq, nr_irqs, arg);----調用struct irq_domain中的alloc回調函數進行硬件中斷號和軟件中斷號的映射。if (ret < 0) {mutex_unlock(&irq_domain_mutex);goto out_free_irq_data;}for (i = 0; i < nr_irqs; i++)irq_domain_insert_irq(virq + i);mutex_unlock(&irq_domain_mutex);return virq; ... }int __ref __irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node,struct module *owner) { ...mutex_lock(&sparse_irq_lock);start = bitmap_find_next_zero_area(allocated_irqs, IRQ_BITMAP_BITS,from, cnt, 0);-------------------在allocated_irqs位圖中查找第一個連續cnt個為0的比特位區域。 ...bitmap_set(allocated_irqs, start, cnt);-------------bitmap_set()設置這些比特位,表示這些比特位已經被占用。mutex_unlock(&sparse_irq_lock);return alloc_descs(start, cnt, node, owner);--------這里要看是否定義了CONFIG_SPARSE_IRQ,如果定義了需要動態分配一個struct irq_desc數據結構,以Radix Tree方式存儲;沒有的話則從irq_desc全局變量中加上偏移即可。err:mutex_unlock(&sparse_irq_lock);return ret; }irq_domain_alloc_irqs_recursive()會根據實際情況決定中斷控制器的遞歸處理,
static int irq_domain_alloc_irqs_recursive(struct irq_domain *domain,unsigned int irq_base,unsigned int nr_irqs, void *arg) {int ret = 0;struct irq_domain *parent = domain->parent;bool recursive = irq_domain_is_auto_recursive(domain);BUG_ON(recursive && !parent);if (recursive)ret = irq_domain_alloc_irqs_recursive(parent, irq_base,nr_irqs, arg);if (ret >= 0)ret = domain->ops->alloc(domain, irq_base, nr_irqs, arg);if (ret < 0 && recursive)irq_domain_free_irqs_recursive(parent, irq_base, nr_irqs);return ret; }至此完成了中斷DeviceTree的解析,各數據結構的初始化,以及最主要的硬件中斷號到Linux中斷號的映射。
3. ARM底層中斷處理
ARM底層中斷處理的范圍是從中斷異常觸發,到irq_handler。
3.1 中斷硬件行為
外設有事件需要報告SoC時,通過和SoC鏈接的中斷管腳發送中斷信號,可能是邊沿觸發信號也可能是電平觸發信號。
中斷控制器會感知中斷信號,中斷控制器仲裁單元選擇優先級最高的中斷發送到CPU Interface,CPU Interface決定將中斷分發到哪個CPU核心。
GIC控制器和CPU核心之間通過一個nIRQ(IRQ request input line)信號來通知CPU。
CPU核心感知到中斷發生之后,硬件會做如下工作:
- 保存中斷發生時CPSR寄存器內容到SPSR_irq寄存器中
- 修改CPSR寄存器,讓CPU進入處理器模式(processor mode)中的IRQ模式,即修改CPSR寄存器中的M域設置為IRQ Mode。
- 硬件自動關閉中斷IRQ或FIQ,即CPSR中的IRQ位或FIQ位置1。------------硬件自動關中斷
- 保存返回地址到LR_irq寄存器中。
- 硬件自動調轉到中斷向量表的IRQ向量。-------------------------------------------從此處開始進入軟件領域
當從中斷返回時需要軟件實現如下操作:
- 從SPSR_irq寄存器中恢復數據到CPSR中。
- 從LR_irq中恢復內容到PC中,從而返回到中斷點的下一個指令處執行。
3.2 中斷異常向量
3.2.1 中斷異常向量代碼段初始化
內核編譯時,異常向量表存放在可執行文件的__init段中:arch/arm/kernel/vmlinux.lds.S。
__vectors_start和__vectors_end指向vectors段的開始和結束地址,__stubs_start和__stubs_end存放異常向量stubs代碼段。兩者都是頁面對齊,大小都為一個頁面。
__vectors_start = .;.vectors 0 : AT(__vectors_start) {*(.vectors)----------------------------------保存.vectors段數據}. = __vectors_start + SIZEOF(.vectors);__vectors_end = .;__stubs_start = .;.stubs 0x1000 : AT(__stubs_start) {*(.stubs)------------------------------------存放.stubs段數據}. = __stubs_start + SIZEOF(.stubs);__stubs_end = .;系統初始化時會把上述兩個段復制到高端地址處,即ixffff_0000:start_kernel->setup_arch->paging_init->devicemap_init。
static void __init devicemaps_init(const struct machine_desc *mdesc) {struct map_desc map;unsigned long addr;void *vectors;/** Allocate the vector page early.*/vectors = early_alloc(PAGE_SIZE * 2);-------------------------------分配兩個頁面用于映射到high vectors高端地址。early_trap_init(vectors);-------------------------------------------實現異常向量表的復制動作。.../** Create a mapping for the machine vectors at the high-vectors* location (0xffff0000). If we aren't using high-vectors, also* create a mapping at the low-vectors virtual address.*/map.pfn = __phys_to_pfn(virt_to_phys(vectors));---------------------vectors物理頁面號map.virtual = 0xffff0000;-------------------------------------------待映射到的虛擬地址0xffff_0000~0xffff_0fffmap.length = PAGE_SIZE;---------------------------------------------映射區間大小 #ifdef CONFIG_KUSER_HELPERSmap.type = MT_HIGH_VECTORS;-----------------------------------------映射到high vector #elsemap.type = MT_LOW_VECTORS; #endifcreate_mapping(&map);if (!vectors_high()) {map.virtual = 0;map.length = PAGE_SIZE * 2;map.type = MT_LOW_VECTORS;create_mapping(&map);}/* Now create a kernel read-only mapping */map.pfn += 1;map.virtual = 0xffff0000 + PAGE_SIZE;------------------------------映射到0xffff_1000~0xffff_1ffffmap.length = PAGE_SIZE;map.type = MT_LOW_VECTORS;create_mapping(&map); ... }early_trap_init分別將__vectors_start和__stubs_start兩個頁面復制到分配的兩個頁面中。
void __init early_trap_init(void *vectors_base) { ...unsigned long vectors = (unsigned long)vectors_base;extern char __stubs_start[], __stubs_end[];extern char __vectors_start[], __vectors_end[];unsigned i;vectors_page = vectors_base;/** Poison the vectors page with an undefined instruction. This* instruction is chosen to be undefined for both ARM and Thumb* ISAs. The Thumb version is an undefined instruction with a* branch back to the undefined instruction.*/for (i = 0; i < PAGE_SIZE / sizeof(u32); i++)((u32 *)vectors_base)[i] = 0xe7fddef1;---------------------------第一個頁面全部填充未定義指令0xe7fddef1。/** Copy the vectors, stubs and kuser helpers (in entry-armv.S)* into the vector page, mapped at 0xffff0000, and ensure these* are visible to the instruction stream.*/memcpy((void *)vectors, __vectors_start, __vectors_end - __vectors_start);memcpy((void *)vectors + 0x1000, __stubs_start, __stubs_end - __stubs_start); ... }3.2.2 中斷異常向量
中斷發生后,軟件跳轉到中斷向量表開始vector_irq執行,vector_irq在結尾的時候根據中斷發生點所在模式,決定跳轉到__irq_usr或者__irq_svc。
vector_irq在arch/arm/kernel/entry-armv.S由宏vector_stub定義。
關于correction==4,需要減去4字節才是返回地址?
vector_stub宏參數correction為4,。
正在執行指令A時發生了中斷,由于ARM流水線和指令預取等原因,pc指向A+8B處,那么必須等待指令A執行完畢才能處理該中斷,這時PC已經更新到A+12B處。
進入中斷響應前夕,pc寄存器的內容被裝入lr寄存器中,lr=pc-4,即A+8B地址處。
因此返回時要pc=lr-4,才是被中斷時要執行的下一條指令。所以lr要回退4B。
.section .vectors, "ax", %progbits __vectors_start:W(b) vector_rstW(b) vector_undW(ldr) pc, __vectors_start + 0x1000W(b) vector_pabtW(b) vector_dabtW(b) vector_addrexcptnW(b) vector_irq---------------------------------------------------------------跳轉到vector_irqW(b) vector_fiq/** Interrupt dispatcher*/vector_stub irq, IRQ_MODE, 4------------------------------------------------vector_stub宏定義了vector_irq.long __irq_usr @ 0 (USR_26 / USR_32).long __irq_invalid @ 1 (FIQ_26 / FIQ_32).long __irq_invalid @ 2 (IRQ_26 / IRQ_32).long __irq_svc @ 3 (SVC_26 / SVC_32)----------------------------svc模式數值是0b10011,與上0xf后就是3。.long __irq_invalid @ 4.long __irq_invalid @ 5.long __irq_invalid @ 6.long __irq_invalid @ 7.long __irq_invalid @ 8.long __irq_invalid @ 9.long __irq_invalid @ a.long __irq_invalid @ b.long __irq_invalid @ c.long __irq_invalid @ d.long __irq_invalid @ e.long __irq_invalid @ f3.3 內核空間中斷處理__irq_svc
__irq_svc處理發生在內核空間的中斷,主要svc_entry保護中斷現場;irq_handler執行中斷處理;如果打開搶占功能,檢查是否可以搶占;最后svc_exit執行中斷退出處理。
__irq_svc:svc_entryirq_handler#ifdef CONFIG_PREEMPT-----------------------------------------------------中斷處理結束后,發生搶占的地方?get_thread_info tskldr r8, [tsk, #TI_PREEMPT] @ get preempt count--------------獲取thread_info->preempt_cpunt變量;preempt_count為0,說明可以搶占進程;preempt_count大于0,表示不能搶占。ldr r0, [tsk, #TI_FLAGS] @ get flags------------------------獲取thread_info->flags變量teq r8, #0 @ if preempt count != 0movne r0, #0 @ force flags to 0tst r0, #_TIF_NEED_RESCHED-----------------------------------------判斷是否設置了_TIF_NEED_RESCHED標志位blne svc_preempt #endifsvc_exit r5, irq = 1 @ return from exceptionUNWIND(.fnend ) ENDPROC(__irq_svc)svc_entry將中斷現場保存到內核棧中,主要是struct pt_regs中的寄存器。
.macro svc_entry, stack_hole=0, trace=1UNWIND(.fnstart )UNWIND(.save {r0 - pc} )sub sp, sp, #(S_FRAME_SIZE + \stack_hole - 4) #ifdef CONFIG_THUMB2_KERNELSPFIX( str r0, [sp] ) @ temporarily savedSPFIX( mov r0, sp )SPFIX( tst r0, #4 ) @ test original stack alignmentSPFIX( ldr r0, [sp] ) @ restored #elseSPFIX( tst sp, #4 ) #endifSPFIX( subeq sp, sp, #4 )stmia sp, {r1 - r12}ldmia r0, {r3 - r5}add r7, sp, #S_SP - 4 @ here for interlock avoidancemov r6, #-1 @ "" "" "" ""add r2, sp, #(S_FRAME_SIZE + \stack_hole - 4)SPFIX( addeq r2, r2, #4 )str r3, [sp, #-4]! @ save the "real" r0 copied@ from the exception stackmov r3, lr@@ We are now ready to fill in the remaining blanks on the stack:@@ r2 - sp_svc@ r3 - lr_svc@ r4 - lr_<exception>, already fixed up for correct return/restart@ r5 - spsr_<exception>@ r6 - orig_r0 (see pt_regs definition in ptrace.h)@stmia r7, {r2 - r6}.if \trace #ifdef CONFIG_TRACE_IRQFLAGSbl trace_hardirqs_off #endif.endif.endmsvc_exit準備返回中斷現場,然后通過ldmia指令從棧中恢復15個寄存器,包括pc內容,至此整個中斷完成并返回。
.macro svc_exit, rpsr, irq = 0...msr spsr_cxsf, \rpsrldmia sp, {r0 - pc}^ @ load r0 - pc, cpsr.endmirq_handler進入高層中斷處理。
4. 高層中斷處理
irq_handler匯編宏是ARCH層和高層中斷處理分割線,在這里從匯編跳轉到C進行GIC相關處理。
前面介紹了一個中斷是如何從硬件中斷號映射到Linux中斷號的,那么當一個中斷產生后它從應將到軟件識別中斷號,再到轉換成Linux中斷號是什么路徑呢?
這里就從irq_handler開始分析流程:
irq_handler() ->handle_arch_irq()->gic_handle_irq() ->handle_domain_irq()->__handle_domain_irq()-------------讀取IAR寄存器,響應中斷,獲取硬件中斷號 ->irq_find_mapping()------------------------------------------------將硬件中斷號轉變成Linux中斷號 ->generic_handle_irq()---------------------------------------------之后的操作都是Linux中斷號 ->handle_percpu_devid_irq()-----------------------------------SGI/PPI類型中斷處理 ->handle_fasteoi_irq()--------------------------------------------SPI類型中斷處理 ->handle_irq_event()->handle_irq_event_percpu()------執行中斷處理核心函數 ->action->handler-----------------------------------------------執行primary handler。 ->__irq_wake_thread()----------------------------------------根據需要喚醒中斷內核線程
4.1 irq_handler
irq_handler宏調用handle_arch_irq函數,這個函數set_handle_irq注冊,GICv2對應gic_handle_irq。
.macro irq_handler #ifdef CONFIG_MULTI_IRQ_HANDLERldr r1, =handle_arch_irqmov r0, spadr lr, BSYM(9997f)ldr pc, [r1] #elsearch_irq_handler_default #endif 9997:.endm4.2 gic_handle_irq
git_init_bases設置handle_arch_irq為gic_handle_irq。
void __init gic_init_bases(unsigned int gic_nr, int irq_start,void __iomem *dist_base, void __iomem *cpu_base,u32 percpu_offset, struct device_node *node) { ...if (gic_nr == 0) { ...set_handle_irq(gic_handle_irq);} ... }void __init set_handle_irq(void (*handle_irq)(struct pt_regs *)) {if (handle_arch_irq)return;handle_arch_irq = handle_irq; }gic_handle_irq對將中斷分為兩組:SGI、PPI/SPI。
SGI類型中斷交給handle_IPI()處理;PPI/SPI類型交給handle_domain_irq處理。
static void __exception_irq_entry gic_handle_irq(struct pt_regs *regs) {u32 irqstat, irqnr;struct gic_chip_data *gic = &gic_data[0];void __iomem *cpu_base = gic_data_cpu_base(gic);do {irqstat = readl_relaxed(cpu_base + GIC_CPU_INTACK);---讀取IAR寄存器,表示響應中斷。irqnr = irqstat & GICC_IAR_INT_ID_MASK;-----------------GICC_IAR_INT_ID_MASK為0x3ff,即低10位,所以中斷最多從0~1023。if (likely(irqnr > 15 && irqnr < 1021)) {handle_domain_irq(gic->domain, irqnr, regs);continue;}if (irqnr < 16) {---------------------------------------SGI類型的中斷是CPU核間通信所用,只有定義了CONFIG_SMP才有意義。writel_relaxed(irqstat, cpu_base + GIC_CPU_EOI);----直接寫EOI寄存器,表示結束中斷。 #ifdef CONFIG_SMPhandle_IPI(irqnr, regs);----------------------------irqnr表示SGI中斷類型 #endifcontinue;}break;} while (1); }handle_domain_irq調用__handle_domain_irq,其中lookup置為true。
irq_enter顯式告訴Linux內核現在要進入中斷上下文了,在處理完中斷后調用irq_exit告訴Linux已經完成中斷處理過程。
int __handle_domain_irq(struct irq_domain *domain, unsigned int hwirq,bool lookup, struct pt_regs *regs) {struct pt_regs *old_regs = set_irq_regs(regs);unsigned int irq = hwirq;int ret = 0;irq_enter();-----------------------------------------------通過顯式增加hardirq域計數,通知Linux進入中斷上下文#ifdef CONFIG_IRQ_DOMAINif (lookup)irq = irq_find_mapping(domain, hwirq);-----------------根據硬件中斷號找到對應的軟件中斷號 #endif/** Some hardware gives randomly wrong interrupts. Rather* than crashing, do something sensible.*/if (unlikely(!irq || irq >= nr_irqs)) {ack_bad_irq(irq);ret = -EINVAL;} else {generic_handle_irq(irq);--------------------------------開始具體某一個中斷的處理,此處irq已經是Linux中斷號。}irq_exit();-------------------------------------------------退出中斷上下文set_irq_regs(old_regs);return ret; }irq_find_mapping在struct irq_domain中根據hwirq找到Linux環境的irq。
unsigned int irq_find_mapping(struct irq_domain *domain,irq_hw_number_t hwirq) {struct irq_data *data; .../* Check if the hwirq is in the linear revmap. */if (hwirq < domain->revmap_size)return domain->linear_revmap[hwirq];----------------linear_revmap[]在__irq_domain_alloc_irqs()->irq_domain_insert_irq()時賦值。 ... }generic_handle_irq參數是irq號,irq_to_desc()根據irq號找到對應的struct irq_desc。
然后調用irq_desc->handle_irq處理對應的中斷。
int generic_handle_irq(unsigned int irq) {struct irq_desc *desc = irq_to_desc(irq);if (!desc)return -EINVAL;generic_handle_irq_desc(irq, desc);return 0; }static inline void generic_handle_irq_desc(unsigned int irq, struct irq_desc *desc) {desc->handle_irq(irq, desc); }關于desc->handle_irq來歷,在每個中斷注冊的時候,由gic_irq_domain_map根據hwirq號決定。
在gic_irq_domain_map的時候根據hw號決定handle,hw硬件中斷號小于32指向handle_percpu_devid_irq,其他情況指向handle_fasteoi_irq。
void __irq_set_handler(unsigned int irq, irq_flow_handler_t handle, int is_chained,const char *name) { ...desc->handle_irq = handle;desc->name = name; ... }handle_percpu_devid_irq處理0~31的SGI/PPI類型中斷,首先響應IAR,然后執行handler,最后發送EOI。
void handle_percpu_devid_irq(unsigned int irq, struct irq_desc *desc) {struct irq_chip *chip = irq_desc_get_chip(desc);struct irqaction *action = desc->action;void *dev_id = raw_cpu_ptr(action->percpu_dev_id);irqreturn_t res;kstat_incr_irqs_this_cpu(irq, desc);if (chip->irq_ack)chip->irq_ack(&desc->irq_data);trace_irq_handler_entry(irq, action);res = action->handler(irq, dev_id);trace_irq_handler_exit(irq, action, res);if (chip->irq_eoi)chip->irq_eoi(&desc->irq_data);-------------------調用gic_eoi_irq()函數 }irq_enter和irq_exit顯式地處理hardirq域計數,兩者之間的部分屬于中斷上下文。
/** Enter an interrupt context.*/ void irq_enter(void) {rcu_irq_enter();if (is_idle_task(current) && !in_interrupt()) {/** Prevent raise_softirq from needlessly waking up ksoftirqd* here, as softirq will be serviced on return from interrupt.*/local_bh_disable();tick_irq_enter();_local_bh_enable();}__irq_enter();---------------------------------------------顯式增加hardirq域計數 }#define __irq_enter() \do { \account_irq_enter_time(current); \preempt_count_add(HARDIRQ_OFFSET); \----------------顯式增加hardirq域計數trace_hardirq_enter(); \} while (0)void irq_exit(void) { #ifndef __ARCH_IRQ_EXIT_IRQS_DISABLEDlocal_irq_disable(); #elseWARN_ON_ONCE(!irqs_disabled()); #endifaccount_irq_exit_time(current);preempt_count_sub(HARDIRQ_OFFSET);---------------------------顯式減少hardirq域計數if (!in_interrupt() && local_softirq_pending())--------------當前不處于中斷上下文,且有pending的softirq,進行softirq處理。invoke_softirq();tick_irq_exit();rcu_irq_exit();trace_hardirq_exit(); /* must be last! */ }4.2.1 中斷上下文
判斷當前進程是處于中斷上下文,還是進程上下文依賴于preempt_count,這個變量在struct thread_info中。
preempt_count計數共32bit,從低到高依次是:
#define PREEMPT_BITS 8 #define SOFTIRQ_BITS 8 #define HARDIRQ_BITS 4 #define NMI_BITS 1 #define hardirq_count() (preempt_count() & HARDIRQ_MASK)-----------------硬件中斷計數 #define softirq_count() (preempt_count() & SOFTIRQ_MASK)-----------------軟中斷計數 #define irq_count() (preempt_count() & (HARDIRQ_MASK | SOFTIRQ_MASK \----包括NMI、硬中斷、軟中斷三者計數| NMI_MASK))/** Are we doing bottom half or hardware interrupt processing?** in_irq() - We're in (hard) IRQ context* in_softirq() - We have BH disabled, or are processing softirqs* in_interrupt() - We're in NMI,IRQ,SoftIRQ context or have BH disabled* in_serving_softirq() - We're in softirq context* in_nmi() - We're in NMI context* in_task() - We're in task context** Note: due to the BH disabled confusion: in_softirq(),in_interrupt() really* should not be used in new code.*/ #define in_irq() (hardirq_count())----------------------------判斷是否正在硬件中斷上下文 #define in_softirq() (softirq_count())------------------------判斷是否正在處理軟中斷或者禁止BH。 #define in_interrupt() (irq_count())--------------------------判斷是否處于NMI、硬中斷、軟中斷三者之一或者兼有上下文 #define in_serving_softirq() (softirq_count() & SOFTIRQ_OFFSET)---判斷是否處于軟中斷上下文。 #define in_nmi() (preempt_count() & NMI_MASK)-----------------判斷是否處于NMI上下文 #define in_task() (!(preempt_count() & \(NMI_MASK | HARDIRQ_MASK | SOFTIRQ_OFFSET)))------判斷是否處于進程上下文思考:in_softirq()和in_serving_softirq()區別?in_interrupt()和in_task()中關于SOFTIRQ_MASK和SOFTIRQ_OFFSET區別?
4.3 handle_fasteoi_irq
handle_fsteoi_irq處理SPI類型的中斷,將主要工作交給handle_irq_event()。
handle_irq_event_percpu()首先處理action->handler,有需要則喚醒中斷內核線程,執行action->thread_fn。
void handle_fasteoi_irq(unsigned int irq, struct irq_desc *desc) {struct irq_chip *chip = desc->irq_data.chip;raw_spin_lock(&desc->lock);if (!irq_may_run(desc))goto out;desc->istate &= ~(IRQS_REPLAY | IRQS_WAITING);kstat_incr_irqs_this_cpu(irq, desc);/** If its disabled or no action available* then mask it and get out of here:*/if (unlikely(!desc->action || irqd_irq_disabled(&desc->irq_data))) {---如果該中斷沒有指定action描述符或該中斷被關閉了IRQD_IRQ_DISABLED,設置該中斷狀態為IRQS_PENDING,且mask_irq()屏蔽該中斷。desc->istate |= IRQS_PENDING;mask_irq(desc);goto out;}if (desc->istate & IRQS_ONESHOT)----------------------------------------如果中斷是IRQS_ONESHOT,不支持中斷嵌套,那么應該調用mask_irq()來屏蔽該中斷源。mask_irq(desc);preflow_handler(desc);--------------------------------------------------取決于是否定義了freflow_handler()handle_irq_event(desc);cond_unmask_eoi_irq(desc, chip);----------------------------------------根據不同條件執行unmask_irq()解除中斷屏蔽,或者執行irq_chip->irq_eoi發送EOI信號,通知GIC中斷處理完畢。raw_spin_unlock(&desc->lock);return; out:if (!(chip->flags & IRQCHIP_EOI_IF_HANDLED))chip->irq_eoi(&desc->irq_data);raw_spin_unlock(&desc->lock); }handle_irq_event調用handle_irq_event_percpu,執行action->handler(),如有需要喚醒內核中斷線程執行action->thread_fn。
irqreturn_t handle_irq_event(struct irq_desc *desc) {struct irqaction *action = desc->action;irqreturn_t ret;desc->istate &= ~IRQS_PENDING;--------------------------清除IRQS_PENDING標志位irqd_set(&desc->irq_data, IRQD_IRQ_INPROGRESS);---------設置IRQD_IRQ_INPROGRESS標志位,表示正在處理硬件中斷。raw_spin_unlock(&desc->lock);ret = handle_irq_event_percpu(desc, action);raw_spin_lock(&desc->lock);irqd_clear(&desc->irq_data, IRQD_IRQ_INPROGRESS);-------清除IRQD_IRQ_INPROGRESS標志位,表示中斷處理結束。return ret; }irqreturn_t handle_irq_event_percpu(struct irq_desc *desc, struct irqaction *action) {irqreturn_t retval = IRQ_NONE;unsigned int flags = 0, irq = desc->irq_data.irq;do {----------------------------------------------------遍歷中斷描述符中的action鏈表,依次執行每個action元素中的primary handler回調函數action->handler。irqreturn_t res;trace_irq_handler_entry(irq, action);res = action->handler(irq, action->dev_id);---------執行struct irqaction的handler函數。trace_irq_handler_exit(irq, action, res);if (WARN_ONCE(!irqs_disabled(),"irq %u handler %pF enabled interrupts\n",irq, action->handler))local_irq_disable();---------------------------switch (res) {case IRQ_WAKE_THREAD:-------------------------------去喚醒內核中斷線程/** Catch drivers which return WAKE_THREAD but* did not set up a thread function*/if (unlikely(!action->thread_fn)) {warn_no_thread(irq, action);----------------輸出一個打印表示沒有中斷處理函數break;}__irq_wake_thread(desc, action);----------------喚醒此中斷對應的內核線程/* Fall through to add to randomness */case IRQ_HANDLED:-----------------------------------已經處理完畢,可以結束。flags |= action->flags;break;default:break;}retval |= res;action = action->next;} while (action);add_interrupt_randomness(irq, flags);if (!noirqdebug)note_interrupt(irq, desc, retval);return retval; }4.3.1 喚醒中斷內核線程
__irq_wake_thread喚醒對應中斷的內核線程。
void __irq_wake_thread(struct irq_desc *desc, struct irqaction *action) {/** In case the thread crashed and was killed we just pretend that* we handled the interrupt. The hardirq handler has disabled the* device interrupt, so no irq storm is lurking.*/if (action->thread->flags & PF_EXITING)return;/** Wake up the handler thread for this action. If the* RUNTHREAD bit is already set, nothing to do.*/if (test_and_set_bit(IRQTF_RUNTHREAD, &action->thread_flags))--------------若已經對IRQF_RUNTHREAD置位,表示已經處于喚醒中,該函數直接返回。return;desc->threads_oneshot |= action->thread_mask;--------------------thread_mask在共享中斷中,每一個action有一個比特位來表示。thread_oneshot每個比特位表示正在處理的共享oneshot類型中斷的中斷線程。atomic_inc(&desc->threads_active);-------------------------------活躍中斷線程計數wake_up_process(action->thread);---------------------------------喚醒action的thread內核線程 }4.3.2 創建內核中斷線程
irq_thread在中斷注冊的時候,如果條件滿足同時創建rq/xx-xx內核中斷線程,線程優先級是49(99-50),調度策略是SCHED_FIFO。
static int __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new) { .../** Create a handler thread when a thread function is supplied* and the interrupt does not nest into another interrupt* thread.*/if (new->thread_fn && !nested) {struct task_struct *t;static const struct sched_param param = {.sched_priority = MAX_USER_RT_PRIO/2,-------------------------------設置irq內核線程的優先級,在/proc/xxx/sched中看到的prio為MAX_RT_PRIO-1-sched_priority。};t = kthread_create(irq_thread, new, "irq/%d-%s", irq,new->name);--------------------------------------------------創建線程名為irq/xxx-xxx的內核線程,線程執行函數是irq_thread。 ...sched_setscheduler_nocheck(t, SCHED_FIFO, ¶m);----------------------設置進程調度策略為SCHED_FIFO。/** We keep the reference to the task struct even if* the thread dies to avoid that the interrupt code* references an already freed task_struct.*/get_task_struct(t);new->thread = t;-------------------------------------------------------將當前線程和irq_action關聯起來set_bit(IRQTF_AFFINITY, &new->thread_flags);--------------------------對中斷線程設置CPU親和性} ... }4.3.3 內核中斷線程執行
irq_thread是中斷線程的執行函數,在irq_wait_for_interrupt()中等待。
irq_wait_for_interrupt()中判斷IRQTF_RUNTHREAD標志位,沒有置位則schedule()換出CPU,進行睡眠。
直到__irq_wake_thread()置位了IRQTF_RUNTHREAD,并且wake_up_process()后,irq_wait_for_interrupt()返回0。
static int irq_thread(void *data) {struct callback_head on_exit_work;struct irqaction *action = data;struct irq_desc *desc = irq_to_desc(action->irq);irqreturn_t (*handler_fn)(struct irq_desc *desc,struct irqaction *action);if (force_irqthreads && test_bit(IRQTF_FORCED_THREAD,&action->thread_flags))handler_fn = irq_forced_thread_fn;elsehandler_fn = irq_thread_fn;init_task_work(&on_exit_work, irq_thread_dtor);task_work_add(current, &on_exit_work, false);irq_thread_check_affinity(desc, action);while (!irq_wait_for_interrupt(action)) {irqreturn_t action_ret;irq_thread_check_affinity(desc, action);action_ret = handler_fn(desc, action);-----------執行中斷內核線程函數if (action_ret == IRQ_HANDLED)atomic_inc(&desc->threads_handled);----------增加threads_handled計數wake_threads_waitq(desc);------------------------喚醒wait_for_threads等待隊列}/** This is the regular exit path. __free_irq() is stopping the* thread via kthread_stop() after calling* synchronize_irq(). So neither IRQTF_RUNTHREAD nor the* oneshot mask bit can be set. We cannot verify that as we* cannot touch the oneshot mask at this point anymore as* __setup_irq() might have given out currents thread_mask* again.*/task_work_cancel(current, irq_thread_dtor);return 0; }static int irq_wait_for_interrupt(struct irqaction *action) {set_current_state(TASK_INTERRUPTIBLE);while (!kthread_should_stop()) {if (test_and_clear_bit(IRQTF_RUNTHREAD,&action->thread_flags)) {------------判斷thread_flags是否設置IRQTF_RUNTHREAD標志位,如果設置則設置當前狀態TASK_RUNNING并返回0。此處和__irq_wake_thread中設置IRQTF_RUNTHREAD對應。__set_current_state(TASK_RUNNING);return 0;}schedule();-----------------------------------------換出CPU,在此等待睡眠set_current_state(TASK_INTERRUPTIBLE);}__set_current_state(TASK_RUNNING);return -1; }static irqreturn_t irq_thread_fn(struct irq_desc *desc,struct irqaction *action) {irqreturn_t ret;ret = action->thread_fn(action->irq, action->dev_id);---執行中斷內核線程函數,為request_threaded_irq注冊中斷參數thread_fn。irq_finalize_oneshot(desc, action);---------------------針對oneshot類型中斷收尾處理,主要是去屏蔽中斷。return ret; }irq_finalize_oneshot()對ontshot類型的中斷進行收尾操作。
static void irq_finalize_oneshot(struct irq_desc *desc,struct irqaction *action) {if (!(desc->istate & IRQS_ONESHOT) ||action->handler == irq_forced_secondary_handler)return; again:chip_bus_lock(desc);raw_spin_lock_irq(&desc->lock);/** Implausible though it may be we need to protect us against* the following scenario:** The thread is faster done than the hard interrupt handler* on the other CPU. If we unmask the irq line then the* interrupt can come in again and masks the line, leaves due* to IRQS_INPROGRESS and the irq line is masked forever.** This also serializes the state of shared oneshot handlers* versus "desc->threads_onehsot |= action->thread_mask;" in* irq_wake_thread(). See the comment there which explains the* serialization.*/if (unlikely(irqd_irq_inprogress(&desc->irq_data))) {-----------必須等待硬件中斷處理程序清除IRQD_IRQ_INPROGRESS標志位,見handle_irq_event()。因為該標志位表示硬件中斷處理程序正在處理硬件中斷,直到硬件中斷處理完畢才會清除該標志。raw_spin_unlock_irq(&desc->lock);chip_bus_sync_unlock(desc);cpu_relax();goto again;}/** Now check again, whether the thread should run. Otherwise* we would clear the threads_oneshot bit of this thread which* was just set.*/if (test_bit(IRQTF_RUNTHREAD, &action->thread_flags))goto out_unlock;desc->threads_oneshot &= ~action->thread_mask;if (!desc->threads_oneshot && !irqd_irq_disabled(&desc->irq_data) &&irqd_irq_masked(&desc->irq_data))unmask_threaded_irq(desc);----------------------------------執行EOI或者去中斷屏蔽。out_unlock:raw_spin_unlock_irq(&desc->lock);chip_bus_sync_unlock(desc); }至此一個中斷的執行完畢。
4.4 如何保證IRQS_ONESHOT不嵌套?
5. 注冊中斷
5.1 中斷、線程、中斷線程化
中斷處理程序包括上半部硬件中斷處理程序,下半部處理機制,包括軟中斷、tasklet、workqueue、中斷線程化。
當一個外設中斷發生后,內核會執行一個函數來響應該中斷,這個函數通常被稱為中斷處理程序或中斷服務例程。
上半部硬件中斷處理運行在中斷上下文中,要求快速完成并且退出中斷。
中斷線程化是實時Linux項目開發的一個新特性,目的是降低中斷處理對系統實時延遲的影響。
在LInux內核里,中斷具有最高優先級,只要有中斷發生,內核會暫停手頭的工作轉向中斷處理,等到所有掛起等待的中斷和軟終端處理完畢后才會執行進程調度,因此這個過程會造成實時任務得不到及時處理。
中斷上下文總是搶占進程上下文,中斷上下文不僅是中斷處理程序,還包括softirq、tasklet等,中斷上下文成了優化Linux實時性的最大挑戰之一。
5.2 中斷注冊接口
IRQF_*描述的中斷標志位用于request_threaded_irq()申請中斷時描述該中斷的特性。
IRQS_*的中斷標志位是位于struct irq_desc數據結構的istate成員,也即core_internal_state__do_not_mess_with_it。
IRQD_*是struct irq_data數據結構中的state_use_accessors成員一組中斷標志位,通常用于描述底層中斷狀態。
關于IRQF_ONESHOT特別解釋:必須在硬件中斷處理結束之后才能重新使能中斷;線程化中斷處理過程中保持中斷線處于關閉狀態,直到該中斷線上所有thread_fn執行完畢。
#define IRQF_TRIGGER_NONE 0x00000000 #define IRQF_TRIGGER_RISING 0x00000001---------------------------上升沿觸發 #define IRQF_TRIGGER_FALLING 0x00000002--------------------------下降沿觸發 #define IRQF_TRIGGER_HIGH 0x00000004-----------------------------高電平觸發 #define IRQF_TRIGGER_LOW 0x00000008------------------------------地電平觸發 #define IRQF_TRIGGER_MASK (IRQF_TRIGGER_HIGH | IRQF_TRIGGER_LOW | \IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING)--------四種觸發類型 #define IRQF_TRIGGER_PROBE 0x00000010#define IRQF_SHARED 0x00000080-------------------------------多個設備共享一個中斷號 #define IRQF_PROBE_SHARED 0x00000100-----------------------------中斷處理程序允許sharing mismatch發生 #define __IRQF_TIMER 0x00000200------------------------------標記一個時鐘中斷 #define IRQF_PERCPU 0x00000400-------------------------------屬于某個特定CPU的中斷 #define IRQF_NOBALANCING 0x00000800------------------------------禁止在多CPU之間做中斷均衡 #define IRQF_IRQPOLL 0x00001000------------------------------中斷被用作輪詢 #define IRQF_ONESHOT 0x00002000------------------------------一次性觸發中斷,不允許嵌套。 #define IRQF_NO_SUSPEND 0x00004000---------------------------在系統睡眠過程中不要關閉該中斷 #define IRQF_FORCE_RESUME 0x00008000-----------------------------在系統喚醒過程中必須搶孩子打開該中斷 #define IRQF_NO_THREAD 0x00010000----------------------------表示該中斷不會給線程化 #define IRQF_EARLY_RESUME 0x00020000 #define IRQF_COND_SUSPEND 0x00040000#define IRQF_TIMER (__IRQF_TIMER | IRQF_NO_SUSPEND | IRQF_NO_THREAD)enum {IRQS_AUTODETECT = 0x00000001,-------------------處于自動偵測狀態IRQS_SPURIOUS_DISABLED = 0x00000002,----------------被視為“偽中斷”并被禁用IRQS_POLL_INPROGRESS = 0x00000008,------------------正處于輪詢調用actionIRQS_ONESHOT = 0x00000020,----------------------表示只執行一次,由IRQF_ONESHOT轉換而來,在中斷線程化執行完成后需要小心對待,見irq_finalize_oneshot()。IRQS_REPLAY = 0x00000040,-----------------------重新發送一次中斷IRQS_WAITING = 0x00000080,----------------------處于等待狀態IRQS_PENDING = 0x00000200,----------------------該中斷被掛起IRQS_SUSPENDED = 0x00000800,--------------------該中斷被暫停 };enum {IRQD_TRIGGER_MASK = 0xf,-------------------------該中斷觸發類型IRQD_SETAFFINITY_PENDING = (1 << 8),IRQD_NO_BALANCING = (1 << 10),IRQD_PER_CPU = (1 << 11),IRQD_AFFINITY_SET = (1 << 12),IRQD_LEVEL = (1 << 13),IRQD_WAKEUP_STATE = (1 << 14),IRQD_MOVE_PCNTXT = (1 << 15),IRQD_IRQ_DISABLED = (1 << 16),--------------------該中斷處于關閉狀態IRQD_IRQ_MASKED = (1 << 17),------------------該中斷被屏蔽中IRQD_IRQ_INPROGRESS = (1 << 18),------------------該中斷正在被處理中IRQD_WAKEUP_ARMED = (1 << 19),IRQD_FORWARDED_TO_VCPU = (1 << 20), };struct irqaction是每個中斷的irqaction描述符。
struct irqaction {irq_handler_t handler;-----------primary handler函數指針void *dev_id;----------------傳遞給中斷處理程序的參數void __percpu *percpu_dev_id;struct irqaction *next;irq_handler_t thread_fn;---------中斷線程處理程序的函數指針struct task_struct *thread;----------中斷線程的task_struct數據結構unsigned int irq;----------------Linux軟件中斷號unsigned int flags;--------------注冊中斷時用的中斷標志位,IRQF_*。unsigned long thread_flags;------中斷線程相關標志位unsigned long thread_mask;-------在共享中斷中,每一個action有一個比特位來表示。const char *name;----------------中斷線程名稱struct proc_dir_entry *dir; } ____cacheline_internodealigned_in_smp;request_irq調用request_threaded_irq進行中斷注冊,只是少了一個thread_fn參數。這也是兩則的區別所在,request_irq不能注冊線程化中斷。
irq:Linux軟件中斷號,不是硬件中斷號。
handler:指primary handler,也即request_irq的中斷處理函數handler。
thread_fn:中斷線程化的處理函數。
irqflags:中斷標志位,見IRQF_*解釋。
devname:中斷名稱。
dev_id:傳遞給中斷處理程序的參數。
handler和thread_fn分別被賦給action->handler和action->thread_fn,組合如下:
| 1 | √ | √ | 先執行handler,然后條件執行thread_fn。 |
| 2 | √ | × | 等同于request_irq() |
| 3 | × | √ | handler=irq_default_primary_handler |
| 4 | × | × | 返回-EINVAL |
很多request_threaded_irq()使用第3種組合,irq_default_primary_handler()返回IRQ_WAKE_THREAD,將工作交給thread_fn進行處理。
第2種組合相當于request_irq()。
第4種組合不被允許,因為中斷得不到任何處理。
第1種組合較復雜,在handler根據實際情況返回IRQ_WAKE_THREAD(喚醒內核中斷線程)或者IRQ_HANDLED(中斷已經處理完畢,不需要喚醒中斷內核線程)。
request_threaded_irq()對參數進行檢查之后,分配struct irqaction并填充,然后將注冊工作交給__setup_irq()。
static inline int __must_check request_irq(unsigned int irq, irq_handler_t handler, unsigned long flags,const char *name, void *dev) {return request_threaded_irq(irq, handler, NULL, flags, name, dev); }int request_threaded_irq(unsigned int irq, irq_handler_t handler,irq_handler_t thread_fn, unsigned long irqflags,const char *devname, void *dev_id) { ...if (((irqflags & IRQF_SHARED) && !dev_id) ||-----------------------------共享中斷設備必須傳遞啊dev_id參數來區分是哪個共享外設的中斷(!(irqflags & IRQF_SHARED) && (irqflags & IRQF_COND_SUSPEND)) ||((irqflags & IRQF_NO_SUSPEND) && (irqflags & IRQF_COND_SUSPEND)))return -EINVAL;desc = irq_to_desc(irq);--------------------------------------------------通過Linux中斷號找到對應中斷描述符struct irq_desc。if (!desc)return -EINVAL; ...if (!handler) {if (!thread_fn)return -EINVAL;---------------------------------------------------handler和thread_fn不能同時為NULLhandler = irq_default_primary_handler;--------------------------------沒有設置handler,irq_default_primary_handler()默認返回IRQ_WAKE_THREAD。}action = kzalloc(sizeof(struct irqaction), GFP_KERNEL);-------------------分配struct irqaction,并填充相應成員if (!action)return -ENOMEM;action->handler = handler;action->thread_fn = thread_fn;action->flags = irqflags;action->name = devname;action->dev_id = dev_id;chip_bus_lock(desc);-------------------------------------------------------調用desc->irq_data.chip->irq_bus_lock()進行加鎖保護retval = __setup_irq(irq, desc, action);chip_bus_sync_unlock(desc);if (retval)kfree(action); ...return retval; }5.3 __setup_irq
一張圖
__setup_irq()首先做參數檢查,然后根據需要創建中斷內核線程,這期間處理中斷嵌套、oneshot、中斷共享等問題。
還設置了中斷觸發類型設置,中斷使能等工作。最后根據需要喚醒中斷內核線程,并創建此中斷相關sysfs節點。
/** Internal function to register an irqaction - typically used to* allocate special interrupts that are part of the architecture.*/ static int __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new) {struct irqaction *old, **old_ptr;unsigned long flags, thread_mask = 0;int ret, nested, shared = 0;cpumask_var_t mask;if (!desc)return -EINVAL;if (desc->irq_data.chip == &no_irq_chip)----------------------表示沒有正確初始化中斷控制器,對于GICv2在gic_irq_domain_alloc()中指定chip為gic_chip。return -ENOSYS;if (!try_module_get(desc->owner))return -ENODEV;/** Check whether the interrupt nests into another interrupt* thread.*/nested = irq_settings_is_nested_thread(desc);-----------------對于設置了_IRQ_NESTED_THREAD嵌套類型的中斷描述符,必須指定thread_fn。if (nested) {if (!new->thread_fn) {ret = -EINVAL;goto out_mput;}/** Replace the primary handler which was provided from* the driver for non nested interrupt handling by the* dummy function which warns when called.*/new->handler = irq_nested_primary_handler;} else {if (irq_settings_can_thread(desc))-----------------------判斷該中斷是否可以被線程化,如果沒有設置_IRQ_NOTHREAD表示可以被強制線程化。irq_setup_forced_threading(new);}/** Create a handler thread when a thread function is supplied* and the interrupt does not nest into another interrupt* thread.*/if (new->thread_fn && !nested) {-----------------------------對不支持嵌套的線程化中斷創建一個內核線程,實時SCHED_FIFO,優先級為50的實時線程。struct task_struct *t;static const struct sched_param param = {.sched_priority = MAX_USER_RT_PRIO/2,};t = kthread_create(irq_thread, new, "irq/%d-%s", irq,new->name);-----------------------------------由irq、中斷號、中斷名組成的中斷線程名,處理函數是irq_thread()。if (IS_ERR(t)) {ret = PTR_ERR(t);goto out_mput;}sched_setscheduler_nocheck(t, SCHED_FIFO, ¶m);get_task_struct(t);new->thread = t;set_bit(IRQTF_AFFINITY, &new->thread_flags);}if (!alloc_cpumask_var(&mask, GFP_KERNEL)) {ret = -ENOMEM;goto out_thread;}/** Drivers are often written to work w/o knowledge about the* underlying irq chip implementation, so a request for a* threaded irq without a primary hard irq context handler* requires the ONESHOT flag to be set. Some irq chips like* MSI based interrupts are per se one shot safe. Check the* chip flags, so we can avoid the unmask dance at the end of* the threaded handler for those.*/if (desc->irq_data.chip->flags & IRQCHIP_ONESHOT_SAFE)----------表示該中斷控制器不支持中斷嵌套,所以flags去掉IRQF_ONESHOT。new->flags &= ~IRQF_ONESHOT;raw_spin_lock_irqsave(&desc->lock, flags);old_ptr = &desc->action;old = *old_ptr;if (old) {-----------------------------------------------------old指向desc->action指向的鏈表,old不為空說明已經有中斷添加到中斷描述符irq_desc中,說明這是一個共享中斷。shared=1。 .../* add new interrupt at end of irq queue */do {/** Or all existing action->thread_mask bits,* so we can find the next zero bit for this* new action.*/thread_mask |= old->thread_mask;old_ptr = &old->next;old = *old_ptr;} while (old);shared = 1;}/** Setup the thread mask for this irqaction for ONESHOT. For* !ONESHOT irqs the thread mask is 0 so we can avoid a* conditional in irq_wake_thread().*/if (new->flags & IRQF_ONESHOT) {/** Unlikely to have 32 resp 64 irqs sharing one line,* but who knows.*/if (thread_mask == ~0UL) {ret = -EBUSY;goto out_mask;}new->thread_mask = 1 << ffz(thread_mask);} else if (new->handler == irq_default_primary_handler &&---------非IRQF_ONESHOT類型中斷,且handler使用默認irq_default_primary_handler(),如果中斷觸發類型是LEVEL,如果中斷出發后不清中斷容易引發中斷風暴。提醒驅動開發者,沒有primary handler且中斷控制器不支持硬件oneshot,必須顯式指定IRQF_ONESHOT表示位。!(desc->irq_data.chip->flags & IRQCHIP_ONESHOT_SAFE)) {pr_err("Threaded irq requested with handler=NULL and !ONESHOT for irq %d\n",irq);ret = -EINVAL;goto out_mask;}if (!shared) {-------------------------------------------------非共享中斷情況ret = irq_request_resources(desc);if (ret) {pr_err("Failed to request resources for %s (irq %d) on irqchip %s\n",new->name, irq, desc->irq_data.chip->name);goto out_mask;}init_waitqueue_head(&desc->wait_for_threads);/* Setup the type (level, edge polarity) if configured: */if (new->flags & IRQF_TRIGGER_MASK) {ret = __irq_set_trigger(desc, irq,-------------------調用gic_chip->irq_set_type設置中斷觸發類型。new->flags & IRQF_TRIGGER_MASK);if (ret)goto out_mask;}desc->istate &= ~(IRQS_AUTODETECT | IRQS_SPURIOUS_DISABLED | \IRQS_ONESHOT | IRQS_WAITING);irqd_clear(&desc->irq_data, IRQD_IRQ_INPROGRESS);---------清IRQD_IRQ_INPROGRESS標志位if (new->flags & IRQF_PERCPU) {irqd_set(&desc->irq_data, IRQD_PER_CPU);irq_settings_set_per_cpu(desc);}if (new->flags & IRQF_ONESHOT)desc->istate |= IRQS_ONESHOT;if (irq_settings_can_autoenable(desc))irq_startup(desc, true);else/* Undo nested disables: */desc->depth = 1;/* Exclude IRQ from balancing if requested */if (new->flags & IRQF_NOBALANCING) {irq_settings_set_no_balancing(desc);irqd_set(&desc->irq_data, IRQD_NO_BALANCING);}/* Set default affinity mask once everything is setup */setup_affinity(irq, desc, mask);} else if (new->flags & IRQF_TRIGGER_MASK) { ..}new->irq = irq;*old_ptr = new;irq_pm_install_action(desc, new);/* Reset broken irq detection when installing new handler */desc->irq_count = 0;desc->irqs_unhandled = 0;/** Check whether we disabled the irq via the spurious handler* before. Reenable it and give it another chance.*/if (shared && (desc->istate & IRQS_SPURIOUS_DISABLED)) {desc->istate &= ~IRQS_SPURIOUS_DISABLED;__enable_irq(desc, irq);}raw_spin_unlock_irqrestore(&desc->lock, flags);/** Strictly no need to wake it up, but hung_task complains* when no hard interrupt wakes the thread up.*/if (new->thread)wake_up_process(new->thread);------------------------------如果該中斷被線程化,那么就喚醒該內核線程。這里每個中斷對應一個線程。register_irq_proc(irq, desc);----------------------------------創建/proc/irq/xxx/目錄及其節點。new->dir = NULL;register_handler_proc(irq, new);-------------------------------以action->name創建目錄free_cpumask_var(mask);return 0; ... }irq_setup_forced_threading()判斷是否強制當前中斷線程化,然后對thread_flags置位IRQTF_FORCED_THREAD表示此中斷被強制線程化。
將原來的primary handler弄到中斷線程中去執行,原來的primary handler換成irq_default_primary_handler。
并設置secondary的primary handler指向irq_forced_secondary_handler(),原來的thread_fn移到secondary的中線程中執行。
static int irq_setup_forced_threading(struct irqaction *new) {if (!force_irqthreads)---------------------------------------------如果內核啟動參數包含threadirqs,則支持強制線程化。或者CONFIG_PREEMPT_RT_BASE實時補丁打開,這里也強制線程化。return 0;if (new->flags & (IRQF_NO_THREAD | IRQF_PERCPU | IRQF_ONESHOT))----和線程化矛盾的標志位。return 0;new->flags |= IRQF_ONESHOT;----------------------------------------強制線程化的中斷都置位IRQF_ONESHOT。if (new->handler != irq_default_primary_handler && new->thread_fn) {/* Allocate the secondary action */new->secondary = kzalloc(sizeof(struct irqaction), GFP_KERNEL);if (!new->secondary)return -ENOMEM;new->secondary->handler = irq_forced_secondary_handler;new->secondary->thread_fn = new->thread_fn;new->secondary->dev_id = new->dev_id;new->secondary->irq = new->irq;new->secondary->name = new->name;}/* Deal with the primary handler */set_bit(IRQTF_FORCED_THREAD, &new->thread_flags);new->thread_fn = new->handler;new->handler = irq_default_primary_handler;return 0; }setup_irq()、request_threaded_irq()、request_irq()都是對__setup_irq()的包裹。
request_irq()調用request_threaded_irq(),只是少了thread_fn。
request_thraded_irq()和setup_irq()的區別在于,setup_irq()入參是struct irqaction ,而request_threaded_irq()在內部組裝struct irqaction。
6. 一個中斷的生命
經過上面的分析可以看出一個中斷從產生、執行,到最終結束的流程。這里我們用樹形代碼路徑來簡要分析一下一個中斷的生命周期。
vector_irq()->vector_irq()->__irq_svc()->svc_entry()---------------------------------------------------------------保護中斷現場->irq_handler()->gic_handle_irq()--------------------------------------------具體到GIC中斷控制器對應的就是gic_handle_irq(),此處從架構相關進入了GIC相關處理。->GIC_CPU_INTACK------------------------------------------------讀取IAR寄存器,響應中斷->handle_domain_irq()->irq_enter()--------------------------------------------------------進入硬中斷上下文->generic_handle_irq()->generic_handle_irq_desc()->handle_fasteoi_irq()--------------------根據中斷號分辨不同類型的中斷,對應不同處理函數,這里中斷號取大于等于32。->handle_irq_event()->handle_irq_event_percpu()->action->handler()---------------------------對應到特定中斷的處理函數,即上半部->__irq_wake_thread()-----------------------------------------------------如果中斷函數處理返回IRQ_WAKE_THREAD,則喚醒中斷線程進行處理,但不是立即執行中斷線程。->irq_exit()---------------------------------------退出硬中斷上下文。視情況處理軟中斷。->invoke_softirq()-----------------處理軟中斷,超出一定條件任務就會交給軟中斷線程處理。->GIC_CPU_EOI--------------------------寫EOI寄存器,表示結束中斷。至此GIC才會接收新的硬件中斷,此前一直是屏蔽硬件中斷的。->svc_exit-----------------------------------------------------------------恢復中斷現場從上面的分析可以看出:
- 中斷上半部的處理是關硬件中斷的,這里的關硬件中斷是GIC就不接收中斷處理。直到寫EOI之后,GIC仲裁單元才會重新選擇中斷進行處理。
- 軟中斷運行于軟中斷上下文中,但是仍然是關硬件中斷的,這里需要特別注意,軟中斷需要快速處理并且不能睡眠。
- 不是所有軟中斷都運行于軟中斷上下文中,部分軟中斷任務可能會交給ksoftirqd線程處理。
- 包括IRQ_WAKE_THREAD、ksoftirqd、woker等喚醒線程的情況,都不會在中斷上下文中進行處理。中斷上下文中所做的處理只是喚醒,執行時機交給系統調度。
- 如果要提高Linux實時性,有兩個要點:一是將上半部線程化;另一個是將軟中斷都交給ksoftirqd線程處理。
總結
以上是生活随笔為你收集整理的arm Linux 中断管理机制的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: linux实时信号与sigqueue函数
- 下一篇: 逻辑地址、线性地址、物理地址和虚拟地址初