跨通道对应与多通道类别学习的交互机制
文献类型：学位论文
| 作者 | 孙莹
|
| 答辩日期 | 2023-06 |
| 文献子类 | 博士 |
| 授予单位 | 中国科学院大学 |
| 授予地点 | 中国科学院心理研究所 |
| 其他责任者 | 付秋芳 |
| 关键词 | 类别学习 多通道表征 跨通道迁移 跨通道对应 |
| 学位名称 | 理学博士 |
| 学位专业 | 基础心理学 |
| 其他题名 | The mechanism of the interaction between crossmodal corresbondence and multisensorv category learning |
| 中文摘要 | Category learning refers to the process by which individuals classify cluttered information and experiences into meaningful categories and concepts that allows individuals to flexibly adapt to their environment and take action (Seger & Miller, 2010). Previous researches on category learning have explored how people extract statistical correlations between unisensory representations to acquire new multisensory representations or amodal representations, but few studies have examined how category learning affects the multisensory representations that have been formed. Multisensory or amodal representations refer.坦abstract knowledge representations generated by individuals when integrating or processing information, which are independent of the surface features of stimuli, and independent of sensory modalities. Currently, a large number of studies have shown crossmodal correspondence across stimuli and stimulus dimensions, but it is not clear whether the crossmodal correspondence is based on multisensory representations, and whether and how category learning affects crossmodal correspondence. Therefore, this study combines a category learning task and a crossmodal matching task to investigate the types of representations underlying crossmodal correspondence and the time course of crossmodal transfer, and to reveal the role of category and semantic labeling in the formation of multisensory representations. Study 1 combined category learning and crossmodal transfer tasks to investigate the types of multisensory representations on which crossmodal correspondence is based, and to examine how category knowledge affects multisensory representations already present in the brain. The results of Experiment 1 found that the acquired category knowledge about visual position influenced the categorization of pitch when category learning was performed under the visual only condition, indicating that crossmodal transfer occurred between visual position and auditory pitch. Moreover, the correspondence of category labels for position and pitch in multisensory category learning did not affect the crossmodal correspondence effect of pitch-position. The results of Experiment 2 showed that the acquired category knowledge about visual size did not affect the categorization of pitch, but interestingly, the way that combined category labels for size and pitch in multisensory category learning determined the direction of pitch-size crossmodal correspondence, indicating that pitch-size crossmodal correspondence is not stable and can be affected by category and semantic labels. Experiment 3 balanced the difference in task difficulty between position and size category judgments, and further found that no crossmodal transfer occurred between size and pitch, and the way that category labels of size and pitch corresponded in multisensory category learning influenced the direction of pitch-size correspondence. Experiment 4 provided additional evidence that crossmodal transfer between pitch and size did not occur because there was no stable pitch-size correspondence. Study 2 examined the effects of category learning on multisensory representations by manipulating the amount of training in a multisensory category learning task. The correspondence between pitch and position category labels learned by participants in Experiments 5 and 6 was the opposite of the crossmodal correspondence between pitch and position, with 20 blocks completed at once in Experiment 5, and 12 blocks completed every day for seven days, for a total of 84 blocks in Experiment 6. It was found that in both Experiment 5 and Experiment 6, the pitch-position correspondence between audiovisual stimuli did not affect the acquisition of category labels with opposite pitch and position correspondence across modalities in multisensory category learning, and the accuracy of simultaneous audiovisual presentation during learning was significantly higher than that of separate presentation, indicating the classical multisensory facilitation effect. More importantly, the experimental results showed that prolonged the training of the inverse relationship of category labels could change but could not reverse the direction of pitch-position matching. These results suggest that the pitch-position correspondence tends to be based on structural associations and statistical coupling and semantic mediation effects cannot fully explain this phenomenon. Study 3 used the event-related potential technique to examine the dynamic time course of crossmodal correspondence affecting the transfer of category knowledge across modalities. Experiment 7 replicated the behavioral results of Experiment 1，confirming that when there is pitch-position crossmodal correspondence between audiovisual stimuli, individuals can spontaneously transfer categorization knowledge about visual position to auditory pitch, indicting crossmodal transfer effects of category knowledge. ERP results showed that in the auditory categorization task, auditory stimuli induced a greater N1 effect, and a larger P2 effect was induced in the post-learning than in the pre-learning test. These results suggest that the effect of crossmodal correspondence on the crossmodal transfer of category knowledge emerges in the early feature integration phase rather than the late category judgment phase. This study systematically investigated the effects of crossmodal correspondence on the crossmodal transfer of category knowledge and its time course, and found that crossmodal correspondence is based on modality-independent multisensory representations. Simultaneously, this study also examined the effects of category learning on crossmodal correspondence, and found that multisensory category learning can determine the direction of the pitch-size correspondence, and can change but cannot fully invert the already formed pitch-position correspondence. These findings deepen the understanding of the mechanism of crossmodal correspondence generation, enrich the theory related to multisensory representation, and help to solve the problem of crossmodal transfer in category learning. At the same time, the related research results have implications for the setup of sensory substitution devices. |
| 英文摘要 | 类别学习指个体将杂乱的信息和经验划分为有意义的类别和概念的过程，其使个体可以灵活地适应环境和采取行动(Seger & Miller, 2010)。以往研究已探讨了人们如何在类别学习中提取单通道表征间的统计相关性，从而习得新的多通道表征或非通道特异性(amodal )表征，但是鲜有研究考察类别学习如何影响已形成的多通道表征。多通道表征或非通道特异性表征是指个体在整合或加工信息时生成的抽象知识表征，其不受刺激的表面特征的影响，并且独立于感觉通道。目前，大量研究表明，不同刺激和刺激维度间具有跨通道对应，但有关跨通道对应是否基于多通道表征以及类别学习是否及如何影响跨通道对应尚不清楚。因此，本研究结合类别学习任务和跨通道迁移任务，通过考察类别知识是否可以在具有跨通道对应的刺激间进行跨通道迁移，探讨跨通道对应所基于的表征类型以及跨通道迁移的时间进程;并通过考察类别学习在跨通道对应形成中的作用，揭示类别和语义标签在多通道表征形成中的作用。 研究一结合类别学习和跨通道迁移任务，通过比较不同类型的跨通道对应刺激对类别知识跨通道迁移的影响，探讨跨通道对应基于的多通道表征类型;并通过考察类别学习是否影响跨通道对应，考察类别知识如何影响大脑中已经存在的多通道表征。实验1结果发现，当仅在视觉通道下进行类别学习时，人们习得的有关视觉位置的类别知识会影响对音高刺激的分类，表明视觉位置和听觉音高间发生了跨通道迁移，而且，多通道类别学习中位置和音高的类别标签的对应关系，并不影响音高一位置的跨通道对应效应。实验2结果表明，有关视觉大小刺激的类别知识并不影响音高刺激的分类，但有趣的是，多通道类别学习中大小和音高的类别标签的对应方式，决定了音高一大小跨通道对应的方向，表明音高一大小的跨通道对应并不稳定，会受到类别和语义标签的影响。实验3平衡了位置和大小类别判断的难度后，结果发现大小和音高刺激之间还是没有发生跨通道迁移，且多通道类别学习中大小和音高的类别标签的对应方式影响了音高一大小对应的方向。实验4则进一步补充说明，音高和大小之间没有发生跨通道迁移是因为没有稳定的音高一大小对应关系。 研究二采用多通道类别学习任务，通过操纵学习的程度，考察了多通道类别学习对跨通道对应中多通道表征的影响。在实验5和6中，被试分别学习音高和位置的类别标签的对应关系，其与音高位置跨通道对应关系相反。在实验5中被试一次完成20个区组的学习，在实验6中则分7天每天完成12个区组，共完成84个区组的学习。结果发现，无论是实验5还是实验6中，即使视听刺激间存在音高一位置对应，也不影响个体在多通道类别学习中习得与跨通道对应相反的音高和位置的类别标签，并且在学习过程中视听同时呈现的正确率显著高于单独呈现的正确率，表现出经典的多通道促进效应。重要的是，实验结果还表明，长时间的类别标签反向关系的学习可以改变但无法反转个体对音高一位置的匹配方向。这说明音高一位置对应可能是基于结构联结而存在，仅依据统计祸合和语义中介效应并无法完全解释这一现象。 研究三采用事件相关电位技术，考察跨通道对应影响类别知识跨通道迁移的动态时间进程。实验7重复了实验1的行为实验结果，说明了当视听刺激间具有音高一位置跨通道对应关系时，个体可以自发地将视觉位置信息的分类规则迁移到对听觉音调的分类判断上，表现出类别知识的跨通道迁移效应。ERP结果发现，在听觉分类任务中，听觉刺激在学习前测比学习后测诱发了更大的N1效应，在学习后测比学习前测诱发了更大的P2效应。这些结果表明，跨通道对应对类别知识跨通道迁移的影响出现在早期的特征整合阶段，而非晚期的类别判断阶段。 本研究一方面考察了跨通道对应关系对类别知识跨通道迁移的影响及其时间进程，表明跨通道对应是基于通道独立的多通道表征;另一方面通过考察类别学习对跨通道对应的影响，发现多通道类别学习可以决定音高一大小的跨通道对应，但是仅能改变但无法完全反转已经形成的音高一位置的跨通道对应。这些研究发现加深了对跨通道对应产生机制的理解，丰富了多通道表征的相关理论，并有助于解决类别学习中的跨通道迁移的问题。同时，相关研究成果对于感官替代装置的设置也具有一定的启示。 |
| 语种 | 中文 |
| 源URL | [http://ir.psych.ac.cn/handle/311026/46184]  |
| 专题 | 心理研究所_认知与发展心理学研究室 |
| 推荐引用方式 GB/T 7714 | 孙莹. 跨通道对应与多通道类别学习的交互机制[D]. 中国科学院心理研究所. 中国科学院大学. 2023. |
入库方式： OAI收割
来源：心理研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。